homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google AdWords
Forum Library, Charter, Moderators: buckworks & eWhisper & skibum

Google AdWords Forum

Statistical Significance

WebmasterWorld Senior Member 10+ Year Member

Msg#: 3653505 posted 10:53 am on May 19, 2008 (gmt 0)

How many clicks should a keyword receive before it can be considered statistically significant for purposes of ROI tracking?


Receptional Andy

Msg#: 3653505 posted 11:47 am on May 19, 2008 (gmt 0)

There's no simple answer to this question, other than: it depends on how reliable you need the results to be.

If you have a target conversion rate of 1% then you would likely need in the region of 150-200 clicks without conversion before you could consider disabling the words with an acceptable degree of reliability - depending of course, on what is acceptable to you or not.

The actual total number of clicks is not that relevant - how many you would look to get depends on what your goals are, and how confident you want to be that you have enough data to make decisions based on it. Clearly, if you want a 50% conversion rate you will know much quicker whether those words work for you or not.

Sorry if that sounds a bit vague!


WebmasterWorld Senior Member 10+ Year Member

Msg#: 3653505 posted 11:55 am on May 19, 2008 (gmt 0)

OK, that makes sense. How about a 2% conversion rate? Maybe 100-150 with an acceptable degree of reliability?


5+ Year Member

Msg#: 3653505 posted 12:00 pm on May 19, 2008 (gmt 0)

[URL deleted]

Roughly speaking, it depends how you model it and then the confidence level you're willing to accept (given the CPC you are paying on the keyword in question).

I would agree with the above post and say that for me (avg. CPC ~0.20 and target conversion ~ 2.5%) after 100-150 clicks i would get rid of the keyword.

[edited by: buckworks at 12:23 pm (utc) on May 19, 2008]
[edit reason] URL deleted [/edit]

Receptional Andy

Msg#: 3653505 posted 12:07 pm on May 19, 2008 (gmt 0)

Again, how reliable (what degree of confidence) does it need to be? If you're paying 50 a keyword, the you're likely to err on the side of caution. If your keywords are at the 5p level then you would likely go for a much higher degree of confidence.

I can't give you an easy calculation without it being misleading, because of all the variables involved.

I'd imagine 100-150 clicks with a desired 2% conversion rate would be more than reliable enough for most purposes, if the listings received no conversions at all. I'd say perhaps too reliable, especially if click cost is high.


5+ Year Member

Msg#: 3653505 posted 2:50 pm on May 19, 2008 (gmt 0)

You need to conduct F tests.


10+ Year Member

Msg#: 3653505 posted 3:17 pm on May 20, 2008 (gmt 0)

I've battled with this question for a long time and basically came to a non-scientific conclusion, that it has less to do with click volume than I originally thought.

Say you have two business owners. They both pay for the same keyword that gets 100 clicks over a one week period at a $1 CPC. They each pay $100.

Business owner A, gets 2 conversions at $300 aov yeilding $600 for a 5x ROMS (600 - 100)/100.

Business owner B gets 1 conversion at an aov of $150 for 50% ROMS (150 - 100)/100.

If the question is, how much data is statistically significant enough to take some sort of action, than Business owner A is most likely going to say "Raise my bid."(to what is a different question) While Business owner B will most likely, depending on his margins, say "Eh, lets wait and see, i don't feel there is enough data to say this keyword will, on average over time, perform at this level."

For keywords that don't get any conversions, I've historically gone with the rule that, if your site converts at 1%, then you need at least 100 clicks, and probably 300 to account for other friction/seasonality and unpredictable randomness that will never be known. But again, if the margin on just one widget is < $100, at a $1 CPC, you've already lost. But if your margin is $300, then keep running.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google AdWords
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved