Best way to transition between versions

How to move from an old version to a new version of a website

[NickMNS]

I completing work on big project that I have been doing for a client. We have fully rebuilt the website from A-Z. The content has remained the same but the way the content is delivered and managed is completely new. The new version resides on a new server. The old site was mostly static content and the new version will be dynamically served. The current website gets a lot of traffic and generates good revenue.

The last time I did something like this, with my own website, I simply pulled the plug on the old and plugged it into the new. The transition went pretty well from a technical perspective. There were a few bugs but they were quickly ironed out. But there was something that I did that did not go over well with Google, or it was bad timing. My update happened at the same time as Google's March update. The new site took a nose dive only a few days after going live. Was it the signals from the old site that caused the dive or the design of the new version? No one knows.

This time the stakes are higher. I don't want to cause my client's site to take a plunge in the rankings.

After my experience it was suggested to me by Google that I should have transitioned to the new version. To progressively move from one version to the next.

It is worth mentioning here that the website in question is large in terms of pages, tens of millions of pages.

My initial thought is to handle this like an A/B test. I plan on using a Clouflare Worker to route a small percentage of the traffic to the new version and then progressively ramp up. This will allow me to test the server infrastructure to ensure it can handle the load and test the functioning of the site with real users while still giving me the ability to revert back fully to the old version with a flick of a switch.

My worry and the reason for this post is how does this impact Google?

I have read some of the recommendations for A/B testing like:

• use 302 redirect (temp) not a 301 (permanent),
  
• use rel=canonical tag,
  
• don't let the test run too long [it's not a test and how long is too long?],
  
• don't direct Google to one version and user to the other.
  

Is that it, what else should I be worrying about?

[ergophobe]

You mention 302/301 redirects, so I gather the URLs will be changing. Is that necessary? I assume so, otherwise you would be avoiding that fun.

Do you have any idea how often Google is crawling the pages that account for 90% of your sales/traffic/whatever you measure? How many pages is that?

I've never dealt with anything close to tens of millions of pages, but if you have that many URLs and you are changing URLs, it's going to take some time to get that indexed.

[NickMNS]

Once the changes are implemented there should be no change in url. While I'm transitioning my intention was to put the new pages on a sub-domain something like beta.example.com/page-1 where the existing site resides at www.example.com/page-1 then once we are confident that everything works as expected we will 301 the beta. to the www. and the transformation will be complete.

Do you have any idea how often Google is crawling the pages that account for 90%

Google is always crawling, and there are millions of pages so the crawl never ends.

[justpassing]

While I'm transitioning my intention was to put the new pages on a sub-domain something like beta.example.com/page-1 where the existing site resides at www.example.com/page-1

In my opinion, that will be too many URLs changes and might cause a mess on Google's side, from a ranking point of view.

If you are switching the URLs from www to beta, then beta to www again once you are done, this is like moving a site to a new domain twice. Which will cause lot of re-indexing and recalculation of ranking factors (including page rank), and this twice in a short period of time. And I guess the URLs structure of the site is changing too, otherwise you wouldn't need to have 30x redirections in place.

In the past, I did this kind of switch (within the same domain name too), and it caused a lot of perturbation to the ranking for a whole year, before it get stable again (meaning lot of loss of traffic in between).

Since you said the content itself remains the same, what I would do is :
- make the old and new site to co-exist under the same domain name (no sub domain), if needed you can use a catch-all PHP script which will process all requests and check if the URL goes to an old page, or a new one, to decide what to serve.
- for a while (like a month or two), to add a canonical tag to the old pages to their new version,
- after one month or two, to replace the old pages with 301 redirects,

[NickMNS]

Let try and make this clear, the way I was planning to do this was:
A user types in a URL (or clicks on a search result) specifically www.example.com/page-1 then the worker directs the request to either the original site and shows www.example.com/page-1 or to the new version and shows beta.example.com/page-1. Ideally the url the user sees should remain www.example.com/page-1 in both cases but I'm not sure if I can or should do this. Both versions of the page are canonical 95% identical in terms of content and similar in terms of layout. Most of the changes are technical and back-end changes. But the URL structure for both versions is identical.

The problem is that if Googlebot to www.example.com/page-1 and sees the old version, then returns tomorrow and finds a different version and then returns again to find the old version again isn't that going to cause havoc on indexing (repeated millions of times).

The only way to avoid this is to progressively roll-out pages such that www.example.com/page-1 goes to the new version and then always goes to new version while www.example.com/page-2 continues to show the old version, then once page-2 switches to the new version it continues showing the new version. A roll back would then be difficult but the transition would still be progressive.

[Leosghost]

Given that you don't know why your site dipped,( the changes you made may very well have had nothing to do with it ) I'd go the simple rout and just do a live "switch off" of the "old site system" , and "switch" on of the "new site system" , and forget all the complications of using a "beta" sub domain..any thing goes wrong ( don't see why it should , if all the urls are the same and 95% of the content is unchanged ) and, at least you'll not be wondering which of the variables that you'd have introduced if you go the progressive but complex route that you are thinking about, are responsible..

[NickMNS]

@Leosghost I'am very tempted to flick the switch. But there still needs to be some testing. The site gets a high volume of traffic, any sustained down time or loss of traffic will have significant financial impact. The tech stack is completely new and I am a fairly confident that it should handle the load but not 100% certain. The only way to know for sure is to test it a scale.

I guess the other option is to run short tests where I divert a portion of the traffic for a short time period and then revert back to the old version. Run the test just long enough to collect performance data.

[tangor]

You make the move or not.

Pull the trigger, or don't

You know you (your client, etc) want this change.

Do it.

Toe tipping in the pond is not going to give you the results you need.

But I do understand wanting to be "sure" this won't queer the present site.

Nothing wrong with half-measures, just know you will get half-results.

Aside: This is a good thing or is not. Won't know until you do it.