A/B Price Split Experimentation

Anyone tried it on their site? What did you find?

[bakedjake]

Title/Description says it all.

We're going to be deploying some modifications to one of our sites that will let us do some A/B price splitting. At first, we thought of doing this on some sort of time basis (i.e. try price A for month 1, then price B for month 2), but I think it'll look too much like a sale to the users.

Once concern I have about A/B splitting is having the user come in one day, having price A show up, then having them come back and finding price B instead. We can solve most of that problem with cookies, but it still exists.

Anyone have any experiences to share?

[bakedjake]

No one has done these sorts of price trials before?

[rogerd]

Bakedjake, Amazon had exactly these problems when they tried it - customers ended up comparing prices, & got mad.

Being an ex-catalog guy, I love split run testing. I think it would be worth a test. A month to month test wouldn't work, IMO - too many external factors. One way of splitting that comes to mind is IP address - segregate C blocks into two groups. You'll still have problems - a person who logs in from the office and later wants to place the order from his home AOL account, the buddy who e-mails a "check this deal" note to someone else (who doesn't see the deal), etc. It's doubtful that the occasional problem of this type would be a big deal, though. Major catalogers do this all the time.

[cfx211]

I think Amazon was a little trickier in their price testing. Established customers got higher prices than people who had never bought. The idea being to induce people into becoming first time customers.