Site Re-design with A/B Testing... how to structure it properly

[darkroom]

Hey guys,

My company runs a very high trafficked e-commerce site for which they are doing a re-design. Right now the things are setup this way:

User comes to say for instance example.com, they get a soft popup to visit new site at a sub domain abc.example.com which is blocked in robots.txt and user can switch back to old site anytime.

We now want to move the new site on to the main domain with the soft popup for users to visit old site at abc.example.com and vice-versa.

The new site is a LOT more SEO friendly for mobile and desktop but old site isn't, so obviously we are pushing to go live with it ASAP. But the company wants to stick with the old site as the default site for a few months as it's going to be busy times for them ahead.

Now let's say we do the switch and move new site to example.com and old site to abc.example.com. We redirect all users to abc.example.com (old site with bots blocked) and add a soft popup for users to switch to new site if they want to, which will be on example.com. So by default users will get old site on sub domain with the option to switch and bots will always get the new site on main domain (example.com).

There will also be a cookie deployed for users that if user switches to new site, then they will get new site going forward on example.com.

I just don't want it to look like cloaking. But we are essentially just doing A/B testing and giving users both the options.

Please share your opinon if this is okay from Google's point of view and not get penalized.

Thanks in advance!

[JesterMagic]

Existing Users will probably switch over quicker than you think especially if they search on Google first for the product and then click on the search result to get to your site. In this case if the user has the old site cookie I would make sure the redirect goes to the correct old site page and not the home page.

I would also make sure that Google Bot never gets stuck on the old site (ie it some how gets a cookie placed that it automatically gets redirected to the old site which is then set at no index so Google bot gets confused and leaves)

[RedBar]

A few years ago I did precisely what you are suggesting, never again. Whenever I have to do such a thing these days, and I did one last November, I rely on my redirects and a good 404 page, it's simpler, it's cleaner and you would probably be surprised how quick G et al picks up and ranks the new pages.

You will also sleep better:-)

[darkroom]

Thanks for the feedback guys. Yes when a user comes from Google to the new site/page, they will be redirected to the old site with the same content/product (with the option to switch to new site anytime)

When you say "make sure Google bot never gets stuck on the old site". At the moment we are going to be adding a adding a robots file to old site sub domain to block it from crawling. We'll ensure that Googlebot never gets the old site cookie placed. Is there anything else we need to do to ensure it doesn't get stuck on the old site

So just confirming, all this is totally fine to do from Google's perspective?

[JesterMagic]

If you block Google Bot completely from your old site using robots.txt it should be okay from Google's perspective (in terms of not doing anything bad that could get give you a duplicate content issue, penalty, etc...). You just have to make sure Google Bot doesn't get redirected to the old site by accident. It's amazing what Google Bot can find on your site. If it does then your new site may not get fully indexed.

Are all your URLs the same on the old and new site (except for the sub domain of course)?

[bwnbwn]

I am confused just reading your post. How are the users suppose to know what site they are on.
What happens when I buy an item on old site then go to new site and buy an item?
If your moving to a new platform just DO IT.
BAD IDEA to send a user back and forth. BAD

[darkroom]

Thanks again for the feedback.

@JesterMagic - The URLs are not the same in old and new site, but proper mapping has been done to redirect to same/similar product/page for users

@bmnbwn - I agree the setup is quite messy. The cart has a seamless connection between old and new site, so if you add something in old or new site, it will show on both. Same for login etc.

[darkroom]

@jestermagic or anyone else - Any tips on how to make sure Google Bot doesn't get redirected to the old site?

[darkroom]

Right now we are thinking of identifying by crawler / agent as shown on [support.google.com...]

[JesterMagic]

Robots.txt is enough to prevent indexing by Google of your old site. Google will eventually find this site again but I would not add a redirect back to the new site just use robots.txt.

There will also be a cookie deployed for users that if user switches to new site, then they will get new site going forward on example.com.

When you look for the cookie and set this cookie is where you want to make sure you do not automatically redirect the Googlebot. The user agent is the only way I know to identify the bot (plus all the bots that mimic Googlebot). You could throw in the IP range as well but that can change without notice so I wouldn't.

[tangor]

I suspect there's a bit of fear at making the change.

If you want to embrace the g "mobile first" paradigm, and you are prepared to do so, DO IT.

All the rest described is taking a chance that g will get it wrong and that comes back to bite you.

Whether g can see it or not, you are running duplicate content (different urls) and if g finds that by accident the results can be dire.

Just do it. Once, and all redirects are to the "new' site.

[bwnbwn]

Redirect users to old domain. Why makes no sense AT ALL.
So your going to redirect a person but not a bot. Good luck with that.
If as you say the NEW IMPROVED site is more user friendly MORE SEO friendly then AFTER TESTING (you have tested with 10 to 20 people) and confirmed your correct Lon user friendly.
The only soft popup would be welcome to our updated website.
DO NOT. redirect to old. REDIRECT to your new website per page they requested from search.