Idea - sandbox for publisher ad testing?

Making ad testing easier

[ntp7]

Just a quick idea, but wouldn't it benefit Google and the publishers if they would introduce a kind of sandbox for ad testing, so that introducing a new layout, position or scheme wouldn not impact on current CTR and eCPM figures if it turned out to be wrong?

It just seems to me that currently publishers do take a fairly big risk if they introduce experimental ad blocks or whatever in that it could adversely affect their earnings and ratings over the test period. Or am I missing something?

[larryhatch]

I think that would be a fine idea.

[alika]

isn't this the very purpose of testing?

[alika]

What do you mean by sandbox? That the page itself will not appear if it is a new page, or disappear if it is an old page on the SERPs? Or will the ads simply not appear? How can Google know that the ad layout will work -- and make the ad appear -- or if it is a dud -- and make it disappear? How long will the ads be put on sandbox -- e.g. if in the next 24 hours the ad changes were made live and CTR plummeted, then sandbox will kick in? Are you expecting some sort of standard measures to determine the success of an ad layout change for sandbox -- e.g. if CTR will not increase by at least x% after y days of testing, then the ad must be sandboxed?

The entire purpose of testing is to know if the variables that have been changed will work or not -- if not, then the publisher can simply revert to the old style.

[jomaxx]

I don't quite get the point of what you're proposing. It sounds like what you're saying is that you don't want your "official" CTR to go down if you make temporary changes that don't work out, but who cares about that?

[david_uk]

If you monitor your stats, and keep a close eye on your channels then it's very easy to see if the change has been possibly worthwhile or a complete disaster and put the previous layout back.

The problem with the idea of Google keeping the old ctr, epc etc when changes are made is that you wouldn't know if they had worked or not due to the figures not having changed. I guess what you mean is if the change works then you get the extra money, but if it's a failure Google don't knock any off. Nice idea, but it's never going to happen!

I'd advise strict use of channels for banners in order to track changes and set yourself up a good spreadsheet to analyse the changes.

[ntp7]

Ah, OK sorry, clearly being a bit duh there. So really it's just a matter of suck it and see? How long does it take to judge? One thing I noticed is that it can take a while for the context stuff to kick in and the new ads to really match the content subject matter. I also noticed that adding new ads seemed to throw the relevancy of my old ads completely. Is that just me or a standard feature of the Google tech?

[david_uk]

So really it's just a matter of suck it and see?

Yup :)

How long does it take to judge?

I find that within 24 hours I can see if the idea is worth pursuing. Using the channels data will give a good indication quickly.

One thing I noticed is that it can take a while for the context stuff to kick in and the new ads to really match the content subject matter. I also noticed that adding new ads seemed to throw the relevancy of my old ads completely. Is that just me or a standard feature of the Google tech?

I think if your site is new to adsense, then the targetting can be a bit screwier than normal. I've been with adsense since jan '04, so adsense has some idea about the general theme, and ad targetting usually has some element of sanity to it when I make changes.