Don't put too much faith into Google analytics Experiments

I did a test to see my margin of error.

[Roxster]

My webmaster did one A/B test since to see what the margin of error as my programmer requested. He took the home page and tested it against the exact same home page. Over 36,000 visitors, 15 days of testing, yet a 12% improvement to the duplicated home page with no difference! Feeding more traffic to the better converting page likely plays a role in the difference. It looks like all the prior A/B google experiments may be flawed as well.



Variation

Experiment Visits

Conversions

Conversion Rate

Compare to Original

Probability of Outperforming Original






Original
11,921 1,336 11.21%

0%
0.0%




Same page no difference
24,545 3,077 12.54%

12%
83.3%

[Lexur]

Great.
Now you just need to repeat the test twenty times and become rich without change a line of code. :-)