On Ad Procurement Delays

Posted by on Oct 16, 2015 in Online Advertising

We already know that the online advertising ecosystem consists of advertisers and publishers that are bound together via advertising networks. Publishers have the content along with the users and advertisers have ads that they want viewed by interested users. Networks then bring the nuts and bolts to connect the entities together.

Networks typically divide publishers into two categories: those who have access to ads on the server-side and those that can only access ads on the client side. The former is generally more trusted since the ad network has to rely on the word of the publisher regarding certain characteristics of the impression/click. This may be over simplifying things a little, but an example here is something like the publisher saying to the network via server-side scripts:

“Hey I have a user from IP X which wants to see an ad for content Y”

The network then sends back the appropriate ad. What’s important here is that trust is delegated to the publisher in so far as where the call is coming from.

The client-side approach has publishers embedding script/tags into their sites that automatically call the ad network’s server on behalf of the user that is visiting the publisher’s site. So the user’s browser/agent ends up talking directly to the ad network. As a result, very little trust is delegated to the publisher.

Identifying and proving rogue server-side activity is not trivial. Moreover, it’s very difficult to reproduce and so I generally don’t go down this path much when it comes to writing articles for public consumption. Ultimately one always wants to be able to reproduce results for one’s self.

In this article, we take a look at an interesting case where subtle client-side javascript can be used to encourage accidental clicks on an ad. We’ll discuss the theory followed by the results of a simple experiment and conclude with a live example.

Theory

If you’re a smart, unscrupulous publisher with a healthy dollop of traffic heading in your direction every day then why not impact your bottom line by encouraging people to accidentally click on your ads? Obviously blatantly encouraging people to click on ads is prohibited by the large networks. Google AdSense Terms of Service point to AdSense Program Policies which clearly prohibit this behavior:

Clicks on Google ads must result from genuine user interest. Any method that artificially generates clicks or impressions on your Google ads is strictly prohibited.

But what if the publisher sets up a scenario involving an accidental ad click which has users saying to themselves

“What the??? Oh, silly me, that must have been my fault!”

Now whilst the method I’m going to show you won’t hit 100% of users, or even 50%, with enough traffic it will result in a significant rise in clicks and as a result increased revenue: more traffic = more clicks = more revenue.

We start at the beginning: consider a simple page with an area for content and then an area for ads.

When you load a page like this it’s clear what’s going on: ad at the top, content at the bottom. If you’re interested in seeing more content related to the ad then you click the ad. If you’re interested in the content, well, you just read/view the content.

Now consider a page with links to content.

Same thing applies, but there’s a slight variation on user behavior in so far as wanting more content, i.e., if I want more content related to the content below the ad then I have to click on the links.

So:

– If I want more content related to the ad then I click the ad

– If I want more content related to the content, then I click the content.

In this scenario the user has to click somewhere in order to get more content related to this page.

Let’s step this up a little and add one more dimension to this scenario: time. Assuming the publisher has a client-side relationship with the ad network for their ads, what can happen is that the ad loads just a little bit later than the content. What’s happening behind the scenes is that the content section is already prepared on the server from which you are loading the page, so that loads immediately. But because of the relationship the publisher has with the ad network, the user’s browser is responsible for going to the ad network and getting the ad for the page. This introduces a delay and so the ad itself is just that much slower to load than the rest of the page. If you have a really slow connection, then it can be noticeably longer to load.

Note that the content links stay in the same place after the ad has loaded. There’s an alternative to the content staying in place, and that’s where the content shifts to accommodate a new entry in the page:

This difference is significant. If we already know that progressing further from this page requires a click somewhere on the page, then precisely where you click is crucial for it will determine what you end up viewing shortly thereafter. It sounds trivial, so much so that if we were talking face to face you would not be the first person to say to me

“Yeah but that’s obvious, you just click on what you want”

And that’s the point. Since time is a factor in this scenario, then what you want and where you get it from at time T may change at time T+1.

Let’s go through this slowly:

– Page starts loading, ad is not there

– I decide to click on the first content link

– My brain sends a signal to my finger/hand to move to coordinates (x,y) and click/touch

– As the neurons, nerves and muscle tissue involved in this transaction start doing their thing, the browser is communicating with the ad network to load an ad

– The moment before my body has finished completing the task I assigned it (click/touch there), the very position I assigned it to click/touch suddenly changes underfoot for the browser successfully completed retrieving the ad from the ad network

– Unfortunately there’s more time involved in processing the change and since my brain has already committed to the click it happens before I can tell it ‘don’t click there’ so in the next unit of time I’m left sitting there stupefied as to why I ended up on an advertiser’s page and not the link I know I clicked on

Of course, this may all just be me. Perhaps it’s my neurons that are slow firing and incapable of processing the events in front of me in the sub-second times required to adjust my actions. Now it could also be the case that it’s not just me, in which case there may be something more going on here.

Experiment

To get to the bottom of this I designed a simple experiment: users were asked to go to a page and search for bees. When the results loaded (five pictures of bees), users had to click on the first relevant link. Little did they know that I had a random timer on the page (set between 0 and 1200ms) that when triggered was responsible for inserting ads not related to bees (pictures of birds) in the same manner described above: they displace the existing content forcing it to instantly move further down the page.

Over a period of a few days, I paid for 1,245 Mechanical Turk users to participate in this experiment. The results were quite surprising and challenge the notion that it’s all just me and my inability to quickly process a dynamically updating page. I’ve made the experiment publicly accessible, you can try it out for yourself here.

Results

We partition what users clicked on into seven sections (ad1, ad2, organicResult1, …, organicResult5).

Click distribution over these seven sections is as follows:

results

Since 13.81% of users clicked on the ads, it’s worth noting that this is not just a problem specific to me. In an effort to better understand what it is that I was seeing, and since I recorded the delay that was used for each user, I decided to integrate the click delays into partitions of 50ms along with how many clicks were observed for each section.

partitions

partitions-pic

My takeaways:

– Ad1 and organicResult1 clearly take up the majority of the clicks. Of interest is that clicks on ad1 only become significant at the 50ms interval, i.e., less than 50ms equals a lower probability of a click on the ad. This makes sense, since at this short of a delay the ad pretty much renders in synchrony with the rest of the content on the page.

– From the data we have it appears that the best delays for highest throughput of clicks on ads are 100ms, 400ms, 850ms and 1150ms.

Why is this significant?

This is significant because large publishers who deal with monstrous amounts of traffic know that click through rates for ads are typically quite low, say on the order of three to four percent. Even half percent increments in click through result in a significant lift of revenue. As a result, the delay above could result in a double digit percentage increase in click through rate, which is big money for big publishers.

Who loses?

The advertiser will lose, for they are paying for an ad that a user did not want to see.

The user will lose, for they are whisked away to a landing page of an advertiser that they did not intend to click on.

Discussion

Recall the following from the Google AdSense Terms of Service:

Clicks on Google ads must result from genuine user interest. Any method that artificially generates clicks or impressions on your Google ads is strictly prohibited.

If you’re a publisher on the Google AdSense network, it may be well within your interest to take this policy very seriously. There’s a lot in there. Google is basically saying that they only want to send users who are really interested in the advertisement to the landing page of the advertiser, which is what a good advertising network does (they’re protecting their advertisers). If you’re a good publisher, then you’ll do your utmost to make sure you’re not stepping on anyone’s toes here.

Now if the findings of the little experiment above have any weight at all then it’s clear as day to conclude that delays in procuring an advertisement combined with having the container responsible for housing the advertisement dynamically push itself into a page and displace existing clickable content may conflict with Google’s Terms of Service, for there is a significant probability that a user will accidentally click on an ad. Remember, clicks on Google ads must result from genuine user interest.

It’s really easy to fix this problem, some options:

1. If you’re a premium publisher, you can simply ask for access to the server-side API so that you can get ads and deliver it along-side the content in a zippy manner. Good example is a search engine, note that for the really big ones the ads load at the same time the content does (that’s because they’re prepared server-side — see google.com for a quick example)

2.If server-side access is not an option, then you can setup the container housing the ad as fixed and visible. This way the coordinates of the content that the user decided to click on, remain constant regardless of when the ad appears. We’ve already looked at an example of this setup:

3. “But what if I am using client-side calls and there are no ads that come back from the server? Then the style of my page is compromised for I have a blank space where ads should be?” This is even simpler to fix, if there are no ads then simply have the container dynamically disappear, which is the exact opposite of making the container dynamically appear when there are ads. Of course, coordinates are changing again but at least the user may accidentally click on content and not an advertisement.

Example

Enter the third most popular Web site on earth.

 

Did you see that? YouTube is using client-side calls to ask Google for ads, as a result there are delays. When the browser gets the ad then the container hosting the ad is dynamically pushed into the page which takes the place of clickable content, forcing it further down the page.

In case you missed it, I did the video again but this time simulated a busy computer and a slow internet connection (like on an under-powered tablet that is battling to render a page or a phone with a slow connection, for example).

Look for the red arrow, it’ll let you know when to pay close attention.

 

Still not convinced? Try it for yourself. Better yet, replicate my little experiment above by asking a significant other or a few friends to go to YouTube, search for something specific and click the first relevant link they find.  Assuming they don’t mind you watching their screen, pay close attention and take note of what they may ‘accidentally’ click on 13.81% of the time.

« »