Experimenting with AdSense Experiments

[netmeg]

So I guess AdSense Experiments have been around for a while, but since I use the old interface, I haven't really paid attention.

This morning they put a notification in my Dashboard suggesting a style change for one of my top ad units. Really front and center.

Now, I happen to think that the experiment they suggested (remove the background color and border) would cause that ad to be *too* blended, and might get me into trouble but I was feeling adventurous this morning, so I approved it. Google controls the delivery (which now that I think of it, will be weird because I'm already rotating different color ads in that space)

The last time they emailed me a specific suggestion, they turned out to be right and revenue jumped like within the hour. So I'm willing to give it a try.

We'll see what happens. I have a screen shot of their suggestion, just in case they do decide it's too blended and complain.

Have you tried the experiments yet?

[carminejg3]

@IanTurner What I have tried was like a very light blue. But years ago a dark blue like 000080 would do well, and it seems to be working at least on my site this time again.

I am also playing with changing the color of the link text, since that controls the color of the image google has been putting into the images


@Update: Just looked at the experiment, what seems to happen is for X amount of impressions google will 50-50 split for each test, to build a base. Then when google thinks it found the winner it cranks up the page views for that ad. I believe this is what they call the "one arm bandit approach".

Currently one ad that was up over 295% has dropped a little but is still maintaining a healthy lead. And the ad split is currently 30-70. I'm going to leave this up and let it play out.

[IanTurner]

My original blue was 0000FF (bright and google default I think) - I changed this to 6666FF which is slightly paler.

Haven't tried darkening as far as 000080 yet.

[webcentric]

How do you run an experiment when the entire face of the produce is changing right in the middle of your experiment? It's a rhetorical question. I just cancelled all experiments so I can deal with this latest wave of changes. This got old a long time ago. Either Google can experiment with me or I'll experiment with Google. Both at the same time is madness.

[avalon37]

webcentric you are totally correct. This isn't real testing - there is no statistical significance given all the changes/test they are always running. It's testing for Dummies really.

[netmeg]

Interestingly, I want to end my experiments and start a different one, but the end experiment button/link seems to have disappeared.

edit - never mind, I was looking in the wrong place. They really need a link in the reports back to the experiment admin page.

[carminejg3]

These new ads are terrible lol... they really need to give us more control on the format

I do agree you can't test, except the test plays with colors so its still valid.

(I still don't know why we couldn't say Hey Google find the best combo) and let google play with colors and stuff based on info it has on site behaviors

[Chrispcritters]

(I still don't know why we couldn't say Hey Google find the best combo) and let google play with colors and stuff based on info it has on site behaviors

This is sort of what their suggested experiments are. They have reason to believe that the experiment will result in increased revenue.

I do agree that I'd like something broader than what they suggest.

[carminejg3]

Well I mean. Click here for the ad unit and let google test out different colors. and give them control. The experiments require use to still choose the colors and formatting, which does have an impact on clicks.

[dvduval]

One point of advice you will never get...

Google Adsense is not performing well at this location and you might make more with a direct sale. I mostly use Adsense as filler for positions that are between sales, and it works well for me.

[loner]

I'm not sure why Goo even pretends our experiments or input matters or will produce useful results. It's their game after all. Things go up sometimes, but in the long run it all continually diminishes. My 'hands-off' policy toward tinkering with Adsense has kept the overall drop to a minimum and allowed me to spend that time working on my new career rather than in the fantasy world where I pretend I'm working with the Goo.

[denisl]

However much you want knock Google, the way in which your website visitors consider ads has changed, and is changing. So I assume our sites need to change to get the most out of them. Some sensible experimenting seems like a good thing to me.

[webcentric]

Some sensible experimenting seems like a good thing to me.

The operative word being "sensible." Scientific experiments tend to focus on specific isolated variables in a very controlled environment. They don't normally involve changing all the variables at once or attempting to compare unrelated tests such as comparing elephant breeding habits with the age of moon rocks. One scientist doesn't throw acid into another scientist's petri dish mid-experiment and without telling the other about it unless sabotage is the intended result. Sensible?

[netmeg]

Well this is weird. I'm running an experiment right now (background vs border) and it states that the variation is 74% ahead, but only gives the variation a 22% confidence level. Unless I am misunderstanding how this all works, that doesn't make sense. It's saying the variation is running 74% ahead of the original, but the original has a 78% confidence level of pulling it out. Hunh?

[denisl]

My statistics knowledge is getting rusty, but I think the 22% confidence level means that you can only be 22% confident that the experiment results mean anything (need to run the exp longer to give more accurate data).
It doesnt mean that you can be 78% confidence that the result is wrong.

[netmeg]

Ok. You're probably right, although that's not the way it's worded in their help tools.

[ChanandlerBong]

Is it fair to say you can ignore any advantage that a variation has through better CPC?

Why would variation B get 10% higher CPC just because it has a yellow border instead of a green one? CTR..absolutely, but isn't CPC just a lot of noise?

[netmeg]

Yea, I have been ignoring EPC - more concerned with CTR.

[webcentric]

Yea, I have been ignoring EPC - more concerned with CTR.

Exactly. CTR is an engagement measure and ad style/placement/size/type has a profound effect on CTR. The problem with gauging CTR against these factors is that ad relevance is also a factor impacting on CTR. This is why longer-term experiments are required. Ad relevance fluctuates from day-to-day and it can be easy to misread CTR if you just changed ad design.

Just because your CTR goes up or down right after the change doesn't mean the change was the cause. You have to factor ad relevance into the mix and the only somewhat-reliable way to do that is over time. Ad relevance will fluctuate during that time, allowing you to get a CTR average that can be compared to a similar period prior to the change. Just thought I'd mention this because this has been painfully reinforced in my experiments over the last week or so.

[IanTurner]

And Ad relevance can also change with time of day - for instance if your target market is B2B your ad relevance is likely to be higher during office hours.

I'm definitely in favour of longer term experiments when looking at Adsense revenue.

[MrSavage]

I've just used for the first time, the experiment feature. What I like? Ease of use. The simplicity and lack of heavy lifting is right up my alley. The result is rather astonishing and I bow to the Google on this one. Clearly a winning alternative to what I was using, but I need to wait for the "confidence" level to rise.

Summary and note to self: Use it and why the hell did I wait so long to try it.

[netmeg]

Testing image vs text ads; interesting that most years image ads do quite well for me, but they're lagging way behind in the experiment. PLUS as soon as I implemented the experiment, I get a "recommendation" in my AdSense account that I am missing out on revenue by using a text-only ad unit.

D'oh. Google should be smart enough to realize it's a test. Once again, they need an "Shut up, I meant to do that" flag.

[webcentric]

Something just hit me. So, I wanted to run an experiment testing Enhanced Ads vs Non-Enhanced Ads. I set up the experiment in G a few days ago but when I tried to access the report, I just kept getting an error message. So, I killed the experiment.

Prior to setting up the experiment though, I had switched to Non-Enhanced text ads (Enhanced turned off) and that is where I'm at at this point. Here's what hit me. When I had enhanced features on, Desktop was doing OK but mobile and tablet were way down. Now that I've turned enhanced text ads off, mobile and tablet are way up and desktop is in the toilet.

What's odd it that I've always assumed the enhancements were to improve mobile engagement but the opposite seems to be true. Actually, I think the arrow designed primarily for mobile but I think I'm doing better on mobile without it. The downside is that some enhanced features work better than others on different devices but there's only one option set for them (on or off). So "on" for better Desktop performance or "off" for better mobile or whatever you think works best. There seems to be a bit of poison in either choice so "choose your poison" I guess.

Again, I really wonder if we'll see more granular features for selecting Text enhancements in the near future. I certainly hope so. Then I can run some real tests.

[netmeg]

Is it fair to say you can ignore any advantage that a variation has through better CPC?

Why would variation B get 10% higher CPC just because it has a yellow border instead of a green one? CTR..absolutely, but isn't CPC just a lot of noise?

Actually, now that they've added more options to the Experiments, there are tests you can run that might affect EPC. I'm testing pure text ads only vs display ads only, and I'm definitely seeing a big difference in EPC. (For one thing, display ads have to outbid all the available text ads for the placement) So I take back what I said - it's not just CTR, you can also test to potentially improve EPC.

[ken_b]

I recently gave in and tried one of their suggested experiments to increase income.

They were right.
.

[denisl]

May have a glitch. My first experiment (on backgoud colours) gave a summary and confidence level after a couple of days. Then ran a couple more, and after 6 days it is not giving this info, inspite of around 7,000 impressions and 70 clicks on each. One of these 2 new tests is showing double the income which I would have thought was significant.