We’ve gotten several questions recently about whether website testing—such as A/B or multivariate testing—affects a site’s performance in search results. We’re glad you’re asking, because we’re glad you’re testing! A/B and multivariate testing are great ways of making sure that what you’re offering really appeals to your users.
Website testing is when you try out different versions of your website (or a part of your website), and collect data about how users react to each version.
A/B testing is when you run a test by creating multiple versions of a page, each with its own URL. When users try to access the original URL, you redirect some of them to each of the variation URLs and then compare users’ behaviour to see which page is most effective.
Multivariate testing is when you use software to change differents parts of your website on the fly. You can test changes to multiple parts of a page—say, the heading, a photo, and the ‘Add to Cart’ button—and the software will show variations of each of these sections to users in different combinations and then statistically analyze which variations are the most effective. Only one URL is involved; the variations are inserted dynamically on the page.
Despite what the article says, I would recommend cloaking when running multivariate testing.
I always serve Googlebot the version that the most users are seeing. It may be technically cloaking, but the spirit is in the right place. I believe that it is less likely to get you penalized than randomly showing Googlebot different pages every time it visits.