At what point do you give the "win" to an ad in split testing?

statistical relevance of impressions, CTR

[esllou]

I'm no statistician and this type of thing is really causing me a headache.

Right, so I have 3 ads rotating through in an ad group.

ad 1 - 12 clicks - 1,900 imp - 0.64% CTR
ad 2 - 5 clicks - 1,707 imp - 0.29% CTR
ad 3 - 1 click - 855 imp - 0.11% CTR

is that too early to call? How many impressions (or clicks) do you give split testing? If the next two users on G click ad 3, it flies up to 0.35% and that's why I think it's probably too close to call.

I'm sure this type of statistical question has been discussed before but I couldn't find anything.

[skaye]

Howie Johnson in "adwords for Dummies" says not to do anything until 30 clicks have passed-- in my opinion you want the winner to have 30 clicks and a substantial lead on the last placed ad (which gets knocked out).

[JimmyH]

Definitely too soon to call it. I agree at least 30 clicks are needed. There is a tool here that helps you know the statistical confidence of the test.
[splittester.com...]

If you're tracking it, you should probably consider Cost Per Action. For example, you can have an ad that gets better CT, but much lower conversions down the line which ends up making it a lower performer in the end.

Good luck.

[esllou]

ah right, 30 clicks seems a good rule of thumb. I better go and re-activate a couple of creatives. :)

[LucidSW]

From the CTRs, I think these numbers are from the content network. But let's assume they are search numbers. While 30 clicks may be a good rule of thumb, I look at the number of impressions. I go with 125 but prefer 250. It is obvious that Ad 1 is outperforming the other two. I don't think there is a need to wait for 30 clicks.

If these numbers were indeed from the content network, you would also have to take into account which sites these ads were served on. Not all sites are created equal.