|Testing multiple ads|
Best way to find significance
| 10:35 pm on Jan 11, 2009 (gmt 0)|
I have two questions for the real data geeks out there.
The first one is quite simple, if you select "show better performing ads more" or ad optimize or whatever it's called, are Google showing the best ads more of the time in terms of CTR or conversion % or a mixture of the two? I can't find a clear answer to this.
Secondly and a more difficult question. I've added multiple ads in some adgroups to try lots of different copy approaches and made 2 versions of each (headline A versus headline B) - I'm just not sure which approach to take for testing. Show all ads evenly and then split test A and B or "show better performing ads" and then wait for patterns to emerge. Would love to hear peoples ideas and experiences.
| 3:18 pm on Jan 12, 2009 (gmt 0)|
We can't post urls here, but there's a nice blog post out there that you might want to read - do a google search for
split ad testing you're doing it wrong
and it should come up first. Gave me some good ideas when I first read it, and I go back and reference it from time to time still.
| 4:15 pm on Jan 12, 2009 (gmt 0)|
Netmeg, thanks for that, I'm sitting here grinning like a cheshire cat and kicking myself at the same time. What's really annoying is I should have cottoned on to this when I was doing display url testing recently, as I had a new version of my existing champion which under performed, talk about can't see the wood for the trees.
Couldn't have come at a better time for me, as I have also just got the ability to test ad's through to transaction back too!
| 5:40 pm on Jan 12, 2009 (gmt 0)|
> . . . if you select "show better performing ads more" or ad optimize or whatever it's called, are Google showing the best ads more of the time in terms of CTR or conversion % or a mixture of the two? <
Google looks only at the "effective CPM" that it earns from advertising, so it will never look at your conversion rate or profitability. Instead, it looks only at the CTR (Click-Through Rate) in deciding which of several ads to display.
I'm not sure whether Google's algorithms make any effort to identify any "cross-effects" of ad variations; that is, if you have two ad variations and your main competitor also has two ad variations, and the performance of your ads is different depending on which of the competitor's ads are shown, I don't know whether Google attempts to determine which of the two ad pairs work best together.
But it's always important to recognize that the algorithms behind Google AdWords look almost exclusively at the "revenue generated per pageview." (Google does have some overriding "relevancy" algorithms that may block a more-profitable ad from being shown if it would make the search results appear significantly less relevant to a consumer.)
Google does not use your conversion or ROI/ROAS data when choosing which ads to display.
Suppose you have two ads:
(1) one ad draws a 1.0% CTR with a 10% conversion rate (10 conversions per 10,000 adviews),
(2) the second ad draws a 1.2% CTR with a 1% conversion rate (1.2 conversions per 10,000 adviews).
Google will show the first ad more often, even though the second ad is MUCH more profitable for your company. It's up to you to monitor ad performance and disable the first ad, in this example.
| 5:42 pm on Jan 12, 2009 (gmt 0)|
Oops. I meant to write that Google would show the SECOND ad (with the 1.2% CTR) more often than the FIRST ad (with the 1.0% CTR).
| 10:52 pm on Jan 12, 2009 (gmt 0)|
Wow thanks netmeg for that article, I should have also realized that.
OT: I use a spreadsheet that someone told me about on www, basically it takes into clicks/imps/conversions to give you a confidence interval. If you search for,
Statistical Analysis Calculator filetype:xls
it should be the first result. I use this to figure out if I need to collect more data or not.
| 11:43 pm on Jan 14, 2009 (gmt 0)|
Please be careful with the statistical calculators you find around the web. Use common sense.
Think about it this way, if 1-2 more people came to your site and converted, would it totally change the calculations?
If yes, I just can't consider that significant enough. 2 visits shouldn't completely change your confidence levels.