Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

Minimum sample size for testing.

any math / stats whizzes out there?

5:38 pm on Jan 31, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 13, 2003
votes: 0


Here's the basic idea:

I'm running several PPC ad campaigns and trying to determine the optimum bid price.

My question is this: what sample size must I take to make sure that the numbers are statistically significant?

Here's an example:

6000 impressions
52 clicks ( gives a .8% CTR roughly)
2 conversions (3.85%)

Now,I'm not really concerned about the CTR, but the actual conversion ratio.

Most of are successful campaigns are seeing around 9-10% conversion. I want to be able to test for 1-2 weeks, and know with 95% certainty that the conversion rate I see will be within say, 1% (maybe .5%) of the results. How many samples (clicks) must I record to find this?


4:16 am on Feb 8, 2006 (gmt 0)

Full Member

10+ Year Member

joined:Feb 17, 2004
votes: 0

If you run what is known as a monte carlo analysis
(statistical simulation) 200 samples is usually enough.
However, in the case of marketing there can be delays
from click to conversion. I have found that after 1000 clicks statistics start to converge to a realistic number. In some cases I have noticed a good conversion rate for new ads and then after running the ad for a while it slows down.

2:23 pm on Feb 9, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:May 23, 2004
votes: 0

1000 clicks?

I have about 450 keywords in my Overture account, some bids are around 50p per click. You mean I'd have to spend 500 on 1 keyword before seeing whether it's converting or a waste of money? Wouldnt 100 clicks or less be enough?


2:22 pm on Feb 12, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 16, 2006
votes: 0

A good rule of thumb in any statistical process is that the 'noise' will be plus/minus the square root of the total events.

To get within 1%, you need root(n)/n = 1/100.
So n = 10,000 conversions.

You simply don't have the numbers to get that sort of statistical confidence within the time frame.

9:45 pm on Feb 17, 2006 (gmt 0)

New User

5+ Year Member

joined:Feb 13, 2006
votes: 0

Seasonality play a role so when you run a campaign is also something to consider (unless it's not a seasonal product). I think it's probably safe to assume that you'd get far fewer conversions two months away from Christmas as opposed to two weeks away.

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members