Competing Adsense (and AdX) against other networks in DFP

[NathanT]

Found lots of good info here over the years; just registered today.

I am starting to think that I have been evaluating ad networks (and comparing them to Adsense) incorrectly. However, I'm not sure there is a better way to do it, since every option seems flawed.

What I have done in the past when I was presented with an ad network that seemed credible - Criteo, Lijit, etc. - is to ad them to one of our ad units and have them compete with Adsense. Then over time I would update the CPM to match what they were offering. In essentially all cases, that CPM would drop over time until it was negligible, and I determined that we were better off with straight Adsense.

However, I was thinking about this some more, and realized that however good a network is, this result is inevitable when competing an average CPM against a dynamic CPM. Here's an illustration. Let's say that every ad impression has a "true" value x. And to pick a number, let's say Adsense pays us 0.6x for each impression. Now, say there's an awesome ad network that manages to extract the full value, and pays us x per impression. (We don't know that yet though, since we are evaluating that network.) Obviously the ideal result would be that we send all impressions to that network and none to adsense. But what actually happens?

Say the x value of your impressions ranges from 0 to M. If we give the new network a sampling of all our impressions, it should result in an average CPM of M/2. Now we turn on adsense competition. Any impressions where adsense can pay us more than M/2 get sent to adsense. Since adsense only pays us 0.6 of the true value, that means any impressions with value of greater than (M/2) / 0.6, or 0.83M, get sent to adsense - even though the new network would pay more for each of those impressions! Then it gets worse. Since the impressions we're sending to the new network now only range in value from 0 to 0.83M, their new average payout will be just over 0.4M. And now we end up sending impressions to adsense if their value is greater than 0.4M/0.6 = 0.67M, and so on. We keep adjusting the average CPM of the new network down, and adsense keeps taking more impressions, until we end up with essentially all adsense (just as I saw).

So, I'm left to conclude that setting up Adsense (or AdX) to compete in real-time against dynamically-priced networks isn't a good idea. (On the other hand, if you have a lot of fixed-CPM direct advertising, I'm sure it's great.) Anyway, I just did a test of several networks without having them compete against Adsense, and found that some may actually perform very well. Lijit especially is nearly doubling my Adsense CPM in some slots, albeit with a 3x24 frequency cap. Criteo's performance has not been as good, but it still beat Adsense by around 30% with the same frequency cap. Trying a couple others (like Superlinks) but don't have good data back yet. (I also tested a number of backfill networks, including mdotlabs, CPX, and marimedia, but haven't found any that can approach Adsense CPMs.)

One issue is that I'm sure there are instances where Adsense can provide a higher CPM than these networks, even if the network does better on average. By turning off competition, we don't get that. Basically the opposite of the original problem. But since before I always ended up just converging to Adsense alone, this still seems potentially better.

Now, testing isn't finished since those frequency caps also skew things. The next step is to try one network at a time, using Adsense only to serve the passbacks and the remaining impressions once the frequency cap is filled, then look at the combined performance compared to Adsense alone. (Obviously when Adsense is only serving leftover and passback impressions it will have a lower CPM than when it gets everything, so the question will be whether the test network takes enough impressions at a high enough rate that it makes up for this difference.) If I find multiple candidate networks, I could even look at setting up a daisy chain, although I'm not convinced that multiple levels of passbacks are a good idea, given the delay each one causes. (Note that even if you don't daisy-chain passbacks, you can still have multiple networks, since each one can start serving when the next-highest priority one hits its frequency cap. Presumably it would quickly not be worth it though, unless you have a high pageviews/visit.) So I may just send passbacks straight to the backfill level. But basically, the idea is to have one or more taking impressions at the highest priority with frequency caps, from the best-performing to the worst, then have Adsense and AdX competing in real time for the remainder.

Anyway, I'm sure others have run into these same types of questions and problems, so I'm curious how you've dealt with them! Do you agree that having other networks compete with Adsense based on their average CPM doesn't work out? And do you find it's more effective to send passbacks straight to backfill ads, or to daisy chain them? (I've read "experts" recommend both approaches.)

[edited by: martinibuster at 3:12 am (utc) on Jul 1, 2014]
[edit reason] Removed promotional URL. [/edit]

[Chrispcritters]

My reference point here is using DFP and AdSense together.

At a number of events I've spoken with other DFP and AdSense webmasters that have setup their networks to complete (price priority) with a higher CPM than reality, let's say eCPM * 1.25 (or whatever multiplier).

I've also heard of people rounding up their competing networks to the nearest $0.25. So if you had one network (A) paying 1.10 and another (B) at 1.03 you would run both at 1.25. Generally with each additional view of a network towards their frequency cap you'll see a decline in CPM rate. The 1st impression on network (B) with an average CPM rate of 1.03 might pay more than the 3rd impression on network (A).

When it comes to passbacks I don't chain. Depending on the network you may see a 15% loss in impressions tracked. Some more, some less. So with each chain the tracked impressions drop. Could be 50% loss by the time an ad is actually served. Better to get that impression where you can properly track it and make a decision on it.

Of of the things I've done:
1) Create unique adsense channels as passback for each network
2) Keep the "floor" rate of that network with a running CPM over the past x days from the channel.
3) Roll in the revenue from the passback tags for each network into the networks revenue to determine a more accurate CPM rate for the network + passbacks.

[NathanT]

Thanks for the reply. Sounds like you've gone through many of the same thought processes as me.

Using a multiplier for the eCPM makes sense. Effectively that's the same as what I'm doing, except that I essentially use a very large multiplier, so it always wins over Adsense. It's likely that something lower than that would be ideal so that if Adsense is willing to pay a ludicrous amount for a given impression they can still have it. :) I can say from my testing so far though that with good networks like Lijit, I'm doing significantly better (like 30% higher revenue) by giving it 100% of impressions up to its frequency cap, then the rest to Adsense, compared to having it compete at its average CPM (for the reasons I described above). But yes, will likely test having Adsense compete at an inflated level in the future.

Agreed on not daisy-chaining passbacks. I actually tend to see closer to 30% or 40% loss of impressions, which is somewhat mind-boggling to me, but has been consistent across a number of networks. I suppose it would depend a lot on the type of page, browsing habits, etc.

As for your numbered points, some thoughts on those:
1) Totally agree, although instead of making separate channels, I've just used a parameter in DFP, which I include in the passback tags. Then in the DFP reports you can filter for Adsense earnings with that parameter set. (So for instance I'll put googletag.pubads().setTargeting("passback","lijit"); in my passback code.) That also lets me control where passbacks get sent without having to have dedicated ad slots for them.

2) Agreed, although I think we should probably account for lost impressions as well. So if your passback channel has a CPM of $1, but you tend to lose 20% of passback impressions, you should probably set your floor to $0.80 instead.

3) I was originally doing that, but now I'm not so sure. Here's my reasoning. Say Network A pays an average of $1 CPM, and takes 80% of impressions. For simplicity say none of those passbacks are lost, and the passbacks pay $0.50. So, you could either set the eCPM of the network to $1, or to 0.8 * $1 + 0.2 * $0.50 = $0.9. The difference being that any impressions where Adsense is willing to pay more than $0.90 but less than $1 will be taken by Adsense instead of sent to the network. My hypothesis is that those impressions are unlikely to be passed back. The impressions passed back are going to be the 20% 'worst' impressions - bad geos, lack of tracking cookies, etc. And it's unlikely that Adsense will pay $0.90+ for those. So it's likely your total number of passbacks will remain about the same, but the fill rate of Network A will go down, because you're giving some impressions to Adsense at $0.90-$1 that Network A likely would have paid $1 for. Makes sense?

[Chrispcritters]

Make sense.

I segment for geos. So, for Tribal Fusion I worked with them to setup my top x geos and then created geo-specific passback tags and geo-specific floors. I have a very international site so this works for me. You may find that you get better results with some of your networks by just limiting them to US, UK, CA, and AU traffic as a bunch of traffic from Turkey (for example) can drag down the overall CPM rate of the network.

[NathanT]

Ah, I don't envy you that complexity! 95% of our traffic is US, so I have the luxury of basically ignoring geos. I can't imagine having to deal with that as well, since the number of different variables to optimize in advertising is already pretty high.

[Chrispcritters]

Probably still a good idea to geo-target your networks to US traffic if only just in case you do get a burst of non-us traffic. Your networks will appreciate it also -- a little effort on your side can go a long way in getting you favors back later in the game.

[NathanT]

Good point. No harm in explicitly limiting them to US, CA, UK, and AU as you say.

[thedonald123]

Great discussion guys. I love finding discussions about DFP and Adx on webmasterworld, they are so rare compared to Adsense discussions.

I would love to see a separate forum here for medium to large size publishers using DFP and other Ad Networks. There used to be a forum on Burst that was really useful.

One question, both of you keep on mentioning Adsense, are you really referring to Adsense and not Ad Exchange? Why wouldn't you be "upgrading" to Adx if you have enough traffic to bother with Tribal Fusion, Lijit etc.?

I do use daisy chaining, but I limit it as much as I can. So for example Tribal Fusion pays very well if you use DFP geotargeting to only send US, UK, Canada, New Zealand and Australia, different frequency caps for different geo's (what a pain keeping up all those separate line items). And of course need to login every few days to Tribal Fusion and turn off those horrific Tribal Fusion Direct ads. TF will fill about 50% and the rest I send on to Valueclick. I could send it back to to Ad Exchange but what's the point? There is the same discrepancy and chance for lost impressions.

[NathanT]

What I can't understand is, I see lots of people recommending, and saying they have good results with, massive daisy chains. Someone was recommending to me actually doing a double-loop - going through all your networks and adx/adsense at a high price floor, then through them all again with a lower floor. Given the number of impressions we lose with each passback, we'd have none left by the time we got through the second loop! I really am curious where these lost impressions go. It seems like too many to be just due to people leaving the page quickly. And if it was adblock or something like that, it seems like they'd never get the first ad. So it's a mystery. The most likely explanation seems like DFP is dropping the ball, but unfortunately that's very difficult to debug since it's a black box. I would be interested in comparing the results with a different ad server - I've heard that some people use one server for everything else, then just use DFP to compete Adsense and AdX. (So they just use a DFP tag as a creative in their other ad server.) So much work to try though!

We use Adsense and AdX, but we currently have to get AdX through a reseller, since we haven't been invited into the program directly yet (despite having over 30M ad impressions per month). Apparently traffic isn't the only metric they look at when issuing invites, so not everyone gets one, even if their traffic and ad revenue is high. The reseller thing works alright, but it's not perfect, because they take a cut which isn't taken into account when DFP does the competition. So you may have an AdX impression win over an Adsense one, even though after the reseller's cut you would actually make slightly less with AdX. Still, it works out better overall than Adsense alone. (But, at least for us, the impact of adding other networks like Lijit has been larger than that from adding AdX.)

[thedonald123]

Losing impressions with Daisy Chaining was a problem before DFP came into the picture so I doubt you can blame them.

I didn't have a good experience with Lijit although I can't remember much of the details, this was a few years ago though. They didn't have good reporting and I just wasn't impressed.

It's amazing how complicated using an Ad Server is becoming. The more options they give you the more time it takes to keep on top of things. These past few days I've been trying to find help for some questions I have with AdX/DFP and I'm finding all these official help pages and videos for a User Interface that I don't even have yet. It's completely new, everything!

AdX has this option to set a Minimum CPM (It's complicated also because DFP has minimum CPM's also.) and I've been just accepting their Recommended Minimum CPM for Branding and Anonymous ever since I started with AdX 1-2 years ago because that what was what the rep who set me up originally in AdX told me to do, and I didn't have the time or patience to learn what it does. A few days ago I set the price really high on a bunch of Ad Units and I made an extra 40% for a few days, now it's come back down, but I don't know if it's due to the end of the quarter or what.

[NathanT]

The minimum CPM is a price floor; most networks outside of Adsense have this. By setting it lower, you increase the fill rate for that network (fewer passbacks), but decrease your CPM. Raising it is the opposite. One way to handle it is to set the floor equal to the amount you tend to make on your passbacks. (Which you can find by having a parameter in your passback tags, or by using a separate DFP ad unit and/or a separate adsense channel for passbacks.) From there, if you're getting a very good fill rate, you can try increasing the floor and see if you can maintain a good fill rate while increasing your CPM. Alternatively, you can increase your frequency caps to see if you can serve more impressions without dropping your fill rate or CPM too much. (In my OP I actually linked to a good resource I found on this, but a mod removed it thinking it was self-promotion.)

Now, with AdX in DFP it might be a bit different, since I'm assuming it doesn't have to actually pass back to Adsense. Presumably if you set a minimum CPM and it can't hit it, the impression just goes to your next matching line item (likely your remnant ads/Adsense). In that case instead of setting the floor at your passback CPM, you could just set it at your Adsense CPM, if that makes sense. Would love to play with that stuff more, but sadly, not direct AdX access. :(

[thedonald123]

Now, with AdX in DFP it might be a bit different, since I'm assuming it doesn't have to actually pass back to Adsense. Presumably if you set a minimum CPM and it can't hit it, the impression just goes to your next matching line item (likely your remnant ads/Adsense).

Actually, if you set a min CPM for branded then the fill for branded for that Ad Unit goes down but the fill for Anonymous for that Ad Unit goes way up.
If you set a minimum CPM for anonymous as well (much lower than Branded) then I guess AdX can choose to not enter the bidding for that impression, so it should go to my other Ad network line items in DFP (what AdX calls unmatched Ad Requests) , but I'm not seeing a raise in impressions going to my other DFP line items, instead the remnant goes to my final catch all ad unit in DFP (actually an AdX Ad Unit). I've effectively shut those AdX Ad Units out.
A huge problem with the min CPM option in AdX which I don't understand how it can not be dealt with, is that it's across all Geo's which is ridiculous, I mean an impression from India vs. the USA is a 1000% different. Unless I'm supposed to be creating separate ad units for different countries like I do in DFP. Hmmmmm. Wow, I really need to get help on this.
The Branded vs. Anonymous adds an additional layer to the complexity. Only AdX has it, as opposed to Adsense because it's related to RTB, which as I understand is the core difference between Adsense and AdX. Never having been a buyer in an RTB exchange, I don't really have a feel for why there is such a huge difference in RTB between Branding vs Anonymous which doesn't exist in Adsense.
I'm finding it incredibly complicated to figure out the optimal CPM for each Ad Unit, branded and anonymous. In the end I may just go back to using their automated recommendations. I'm just waiting for them to show up again, it takes awhile.