Tracking sales

[thecleaner]

I am using this script to track my sales as I do a/b split tests.

Sometimes the script does not track the sales. This happens about 25-30% of the time where the sale is simply not tracked by the script.

Is this going to have a big effect on the statistical accuracy of rather sales letter A outperforms sales letter B?

I know the numbers want be as big but the *gap* between the best performing copy and the worst, should be the same right?

[ZydoSEO]

It depends... If there is a bug in your script that only triggered for, say test A, then yes it will affect your statistics in that it may give invalid results - say test A converted better, when in fact Test B did (or visa versa). If you can prove that it happens the same percent of the time for test A as for test B then it should not hurt your results relatively speaking. In otherwords, you'll be able to determine which has the better conversion rate, but both conversion rates for Test A & B will be off by the same percentage.

For example:

Test A: 500 customers, 100 sales, 25% didn't get logged, so only 75 sales logged, 20% actual conversion, appears as 15% conversion
Test B: 500 customers, 80 sales, 0% didn't get logged, so all 80 sales logged, 16% actual conversion, appears as 16% conversion

The tests appears to indicate that Test B was better (converted 80 customers instead of 75... However, Test A actually yielded more conversions.

However, in the case:

Test A: 500 customers, 100 sales, 25% didn't get logged, so only 75 sales logged, 20% actual conversion, appears as 15% conversion
Test B: 500 customers, 80 sales, 25% didn't get logged, so only 60 sales logged, 16% actual conversion, appears as 12% conversion

Because the failure to log happens 25% of ALL sales (regardless of test type), the overall results should still be valid if you're trying to determine which test yields a better conversion rate. Test A in the above test can still be determined to be the best performing. You can still see that it performed 25% better than Test B. But this assumes equal distribution (percentage-wise) of the failures across both tests.

[thecleaner]

thank you