A/B Testing and Statistical Significance

[Tonearm]

I have an A/B testing system built into my online store and I'd like to start doing a lot of testing of different things, including prices. At what point have I gathered enough data (orders) to make a decision?

[bcc1234]

Check fisher's exact test or chi square.

[Tonearm]

Thanks, I checked those out.

What about just comparing the number of orders I get while under condition A to the number of orders I get while under condition B?

[Umbra]

I use a 2-tailed t-test assuming equal variance... I hope that's the correct test