Welcome to WebmasterWorld Guest from 54.167.102.69

Forum Moderators: open

Message Too Old, No Replies

# Minimum sample size for testing.

## any math / stats whizzes out there?

#### nquinn

5:38 pm on Jan 31, 2006 (gmt 0)

#### Junior Member

joined:Mar 13, 2003
posts:83

So,

Here's the basic idea:

I'm running several PPC ad campaigns and trying to determine the optimum bid price.

My question is this: what sample size must I take to make sure that the numbers are statistically significant?

Here's an example:

6000 impressions
52 clicks ( gives a .8% CTR roughly)
2 conversions (3.85%)

Now,I'm not really concerned about the CTR, but the actual conversion ratio.

Most of are successful campaigns are seeing around 9-10% conversion. I want to be able to test for 1-2 weeks, and know with 95% certainty that the conversion rate I see will be within say, 1% (maybe .5%) of the results. How many samples (clicks) must I record to find this?

Thanks!

#### TheGuyAboveYou

4:16 am on Feb 8, 2006 (gmt 0)

#### Full Member

joined:Feb 17, 2004
posts:209

If you run what is known as a monte carlo analysis
(statistical simulation) 200 samples is usually enough.
However, in the case of marketing there can be delays
from click to conversion. I have found that after 1000 clicks statistics start to converge to a realistic number. In some cases I have noticed a good conversion rate for new ads and then after running the ad for a while it slows down.

#### mrclark

2:23 pm on Feb 9, 2006 (gmt 0)

#### Junior Member

joined:May 23, 2004
posts:140

1000 clicks?

I have about 450 keywords in my Overture account, some bids are around 50p per click. You mean I'd have to spend £500 on 1 keyword before seeing whether it's converting or a waste of money? Wouldnt 100 clicks or less be enough?

Steve

#### Scruffy

2:22 pm on Feb 12, 2006 (gmt 0)

#### Junior Member

joined:Jan 16, 2006
posts:132

A good rule of thumb in any statistical process is that the 'noise' will be plus/minus the square root of the total events.

To get within 1%, you need root(n)/n = 1/100.
So n = 10,000 conversions.

You simply don't have the numbers to get that sort of statistical confidence within the time frame.

#### wsm2006

9:45 pm on Feb 17, 2006 (gmt 0)

#### New User

joined:Feb 13, 2006
posts:27

Seasonality play a role so when you run a campaign is also something to consider (unless it's not a seasonal product). I think it's probably safe to assume that you'd get far fewer conversions two months away from Christmas as opposed to two weeks away.