Welcome to WebmasterWorld Guest from 54.80.183.100

Forum Moderators: buckworks & eWhisper & skibum

Message Too Old, No Replies

Statistical Significance

     
8:49 pm on Mar 19, 2009 (gmt 0)

New User

5+ Year Member

joined:June 11, 2008
posts:5
votes: 0


How do you all determine whether the results from an A/B test are statistically significant?

I am currently using a spreadsheet that was set up for someone else that spits out some information about the statistical significance, but I'd like to explore this issue in a little more depth...

Thanks!

12:00 pm on Mar 23, 2009 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38167
votes: 63


Well, the first question has to be about the size of the test.

- how many page views did you serve?
- how many unique visitors are there?
- how many actions were taken?

Most of the folks I have seen, have been into the thousands of page views, and atleast 500 uniques. Ultimately, on a smaller scale test - differences less than 10% will have to be explored in greater depth. You will have to make a gut decision. I think if you are wondering if they are significant, it bears further testing.

2:06 pm on Mar 23, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:June 20, 2006
posts:2024
votes: 45

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members