I have about 8000 impressions and 250 clicks or so on each of the ads (obviously the results differ, but that's the average across 6 ads). Is there a rule of thumb on how many clicks/impressions you need before you make optimization decisions?
Any help is appreciated.
thanks.
With 250 clicks you should be near 99%
In general for statistical evaluations the number of impressions delivered is of little relevance relative to the number of clicks.
For example, if you are selling a $25,000 item, with $12,500 proft, and you are spending .05 a click for 250 clicks, you have a small coversion rate but huge ROI if you get a conversion, then 250 clicks is not enough to make a decision.
The 99% number discussed above is not quite right. With 250 clicks you can be 99% confident that the actual number of conversions (in the population of web users) will be + or - 8% . So, if 250 clicks is converting at 9%, you can be 99% confident that the the conversion in the population of web users will be between 1 % and 17%. You could be 95% confident with 250 clicks that the actual number of conversions would be between 3% and 15%. This is assuming randomness, and web users/buyers/clickers are not really random, but whatever.
So, it depends on what confidence level you want and your desired margin of error. Nevertheless, I agree that 30 clicks is often enough to make a decision on, but you will have a large margin of error. If you are dealing with millions of keywords, then 30 clicks on each keyword is plenty. If there is a conversion, keep it, if not get rid of it. Use the law of large numbers to make money, on average.
ss (desired sample size) = Z(squared) * (p) * (1-p)/c(Squared)
where:
Z = Z value (e.g. 1.96 for 95% confidence level)
p = percentage picking a choice, expressed as decimal
(.5 used for sample size needed)
c = confidence interval, expressed as decimal
(e.g., .04 = ±4)
If you wanted to be 80% confident (which I think is often reasonable given the cost), then you would use a 1.28 z value.
In this case, if you have conversion rate of 9% and want to be 80% confident that you are within 5% of what actual conversion would be in the population.
Then you would need a sample size of 1.28(Squared) *.91 *.09 / .05(Squared) = 42.
So, given these numbers, you can be fairly sure (80%) of what your conversion is on only 42 clicks.
The thing is, how willing are you to be wrong? If you are selling that 1000 dollar item, do you want to take a 20% chance on 42 clicks that you did not see a sale due to randomness? Not sure, but if you are betting money and you win 80% of the time and the payout is even odds, then that is a pretty good deal.
Look at the conversion rates for high ticket items. There's a huge difference around pay day times. It's not steady on a day-by-day basis.
In the hotel industry, it's common for bookings to change by days of the week.
Special vacation packages or last minute get-aways are a Thursday/Friday buy - they are not a Monday purchase.
While I'm a huge fan of statistics, understanding how the industry converts is also useful.
[edited by: eWhisper at 2:33 am (utc) on Nov. 1, 2006]
