Welcome to WebmasterWorld Guest from

Forum Moderators: buckworks & eWhisper & skibum

Message Too Old, No Replies

Statistical Significance

8:49 pm on Mar 19, 2009 (gmt 0)

New User

10+ Year Member

joined:June 11, 2008
votes: 0

How do you all determine whether the results from an A/B test are statistically significant?

I am currently using a spreadsheet that was set up for someone else that spits out some information about the statistical significance, but I'd like to explore this issue in a little more depth...


12:00 pm on Mar 23, 2009 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
votes: 90

Well, the first question has to be about the size of the test.

- how many page views did you serve?
- how many unique visitors are there?
- how many actions were taken?

Most of the folks I have seen, have been into the thousands of page views, and atleast 500 uniques. Ultimately, on a smaller scale test - differences less than 10% will have to be explored in greater depth. You will have to make a gut decision. I think if you are wondering if they are significant, it bears further testing.

2:06 pm on Mar 23, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:June 20, 2006
votes: 64


Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members