Welcome to WebmasterWorld Guest from 54.146.194.42

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

Calculating % Chance to Beat Original

     
1:42 pm on Nov 26, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 25, 2003
posts:972
votes: 0


Since Google weaned us off of their amazing multivariate testing tool earlier this year, Iím trying to develop our own in house A/B testing program. In hindsight, I should have done this long ago as a customized solution would have saved tons of time over copying and pasting that code for every test.

Iíve got just about everything we used with Google done now but thereís one statistic they displayed that Iím having trouble figuring out how to calculate, % Chance to Beat Original.

Given the conversion rate and margin of error (using a 95% confidence level) of two variations, how would a person calculate that?

Iím assuming itís not quite as straightforward as simply calculating the difference in means because what youíre really doing is calculating the difference between two normal distrubtions.