| Calculating % Chance to Beat Original
|
woop01
msg:4522581 | 1:42 pm on Nov 26, 2012 (gmt 0) |
Since Google weaned us off of their amazing multivariate testing tool earlier this year, I’m trying to develop our own in house A/B testing program. In hindsight, I should have done this long ago as a customized solution would have saved tons of time over copying and pasting that code for every test.
I’ve got just about everything we used with Google done now but there’s one statistic they displayed that I’m having trouble figuring out how to calculate, % Chance to Beat Original.
Given the conversion rate and margin of error (using a 95% confidence level) of two variations, how would a person calculate that?
I’m assuming it’s not quite as straightforward as simply calculating the difference in means because what you’re really doing is calculating the difference between two normal distrubtions.
|
|
