homepage Welcome to WebmasterWorld Guest from 50.16.130.188
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Advertising / Pay Per Click Engines
Forum Library, Charter, Moderator: open

Pay Per Click Engines Forum

    
is a 70 - 30 split valid?
valid split tests
Jimb0




msg:4662996
 9:39 pm on Apr 14, 2014 (gmt 0)

We are testing 2 landing pages sending 70% to the control and 30% to the challenger. After 3 weeks of testing, the challenger won. Is this statistically significant? Do split tests need to be a true 50 - 50? Why is this or is this not significant?

thanks,

 

RhinoFish




msg:4663565
 4:21 pm on Apr 16, 2014 (gmt 0)

They do not need to be 50/50, but each sample needs to have sufficient size to increase statistical certainty to the level where decisions can be made.

You didn't say the total visits. :-) If it's 7 visits (control) and 3 visits (challenger), neither is large enough to prove anything.

Telling us 3 weeks also doesn't help us, it is the sample size, not the duration over which it's collected (though that can affect a test).

RhinoFish




msg:4663570
 4:34 pm on Apr 16, 2014 (gmt 0)

This is the point where most people ask... how many items do I need in each sample to be meaningful?

Not so simple. It depends on the behavior of what you're measuring and how different the two sets behavior is.

An example:
~~~~~~
A = 1000 clicks, 3 conversions
B = 1100 clicks, 2 conversions
~~~~~~
Inconclusive, not enough conversions, A winning, but only 81% probability. Aim for >95% probability in your tests.

Same ratios of success (A to B), but each one's success rate is a larger set of data:
~~~~~~
100 clicks, 60 conversions
110 clicks, 40 conversions
~~~~~~
A is clear winner, 99%+ probability.

Here's a decent calculator:
[tools.seobook.com...]

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Advertising / Pay Per Click Engines
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved