A/B testing dynamic content and Google

[seoholic]

I want to do some A/B testing on 100k pages with dynamic content and Google Website Optimizer. In Germany I only have to care about Google.

1. Instead of using the Website Optimizer solution for dynamic content, I would like to use a small amount of different javascripts in Website Optimizer to load the content (eg. one script for green text, one for red text, but different texts are always dynamic from the database).

Any known problems with "static JS in Website Optimizer" and "dynamic" content for the bot"?

2. Later I want to test large different pieces of content (eg. 3 Videos vs. 3 related articles) on this pages. The tested content would often be bigger than the whole page was before.

Are there known limitations for the content size in this tests from a SEO perspective?

3. I want to test with as many users as possible.

Are there limitations for the percentage of users used for A/B tests from a SEO perspective?

4. What prevents me from displaying content Google doesn`t like if Google ignores the A/B test as asked in 1;2;3?

[deadsea]

We do tons of A/B experiments on our site (millions of pages). From an SEO perspective, we try to make sure that the titles, meta description, and links on the page for SEO don't get touched by the experiments. Other than that, the experiments are generally run by non-SEO knowledgeable marketing folks. They often do some pretty dramatic changes and we haven't seen any SEO problems because of it.

[seoholic]

Thanks for your feedback.

You mentioned millions of pages: I guess that site size/authority reduces negative impacts.

Additional feedback would be highly appreciated since A/B tests are as essential for revenue as getting traffic.

I did my homework and searched for A/B tests on Webmasterworld:

I have heard of people running into ranking problems when using some kind of home brew split testing. But as you say, not with Google's very own solution. I also assumed that they set up the Website Optimizer technology so it wold not cause problems. Could be a very wrong assumption - we know that one area of Google can make problems for other areas - for example, the first AJAX SERPs broke Google Analytics.

keep test pages out of the Google index.

If the ranking problems did come from Website Optimizer, the clue is probably in the phrase "radically different".

IMO, the best option is to split test away from organic search of any kind. A reliable option is to use pay per click traffic, and robots exclude any pages you use for testing.

My current strategy:
We can't use Google Website Optimizer but we will feed it with some dummy JS just to let Google know about the test.
We will use a different (additional) GA code for each variation.
We will start with one variation(?) sitewide and watch the SERPs for some weeks.
We want to test sitewide to get authentic results.
User behavior is very different depending on the season. So we want to test all variations at once.
We can't afford paid traffic. So we can't get traffic to noindexed pages without destroying the information architecture ("A/B testing" on all internal links? No way).
We want to test IP-based. This should reduce complications with Google.
We want to use our whole traffic for this so we can end the test sooner.

In my case this 100k pages are most of the site and 50% of the traffic and we want to triple the amount of content per page.