Calculating Validity of A/B Split Test

[ivank]

I ran some A/B split tests on some landing pages and was trying to determine the statistical significance of the data.

What are some formulas I can use to determine the level of statistical certainty of the results?

If you've ran A/B split tests in the past, how have you calculated their worth?

Thanks for your help.

[Receptional]

I think you'll be wanting "Student's t-test" for that if your sample size is small (under 30 for each sample). Otherwise, you'd use a "Normal Curve Test".

I assume the "sample" would best be defined as click thrus for example A against Click thrus for example B based on the same number of page impressions.

[edited by: Receptional at 2:03 pm (utc) on Sep. 6, 2006]