Welcome to WebmasterWorld Guest from 23.22.220.37

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

Don't put too much faith into Google analytics Experiments

I did a test to see my margin of error.

     
8:06 pm on Sep 3, 2013 (gmt 0)

Junior Member

5+ Year Member

joined:July 14, 2006
posts: 142
votes: 0


My webmaster did one A/B test since to see what the margin of error as my programmer requested. He took the home page and tested it against the exact same home page. Over 36,000 visitors, 15 days of testing, yet a 12% improvement to the duplicated home page with no difference! Feeding more traffic to the better converting page likely plays a role in the difference. It looks like all the prior A/B google experiments may be flawed as well.



Variation

Experiment Visits

Conversions

Conversion Rate

Compare to Original

Probability of Outperforming Original






Original
11,921 1,336 11.21%

0%
0.0%




Same page no difference
24,545 3,077 12.54%

12%
83.3%
5:56 am on Sept 4, 2013 (gmt 0)

Preferred Member from ES 

10+ Year Member

joined:Nov 13, 2005
posts:646
votes: 0


Great.
Now you just need to repeat the test twenty times and become rich without change a line of code. :-)