Welcome to WebmasterWorld Guest from 220.127.116.11
In a hindsight, it could have been better if you redirected bluewidgets.com to blue-widgets.com instead, and caught type-ins in this way (but this is just a hindsight - do not revert now!)
From the site move point of view, it appears that you have done it all correctly. What I am wondering is whether there are some very strong external links going to blue-widgets.com which may influence Google's behaviour? Also the site is quite large so it may take a long time for Google to process it.
With regards to bluewidgets.com (no hyphen), does this site appears in SERP? And where it does, what happens with the same page from blue-widgets.com, is it also there and if it is, which one ranks higher?
Interesting thing: more than a year ago I've added the SSL to my new one bluewidgets.com. Everything is good, no traffic changes from Google, everybody's happy.So you say your traffic did not suffer after the domain change. Do not compromise this just because Google is still a bit confused with indexing aspects!
Does the old domain still serve a sitemap? Do the old URLs have a "noindex" robots meta? I ask because it can take a long time for Google to digest changes on very large sites as mentioned, but mixed signals might make it take even longer.
Regarding possible duplicate content, yes, there is a period of that and it happens when Google has discovered URL from bluewidgets.com and has not yet recrawled the same URL on blue-widget.com to see the redirect. Until Google requests blue-widget.com, it will not see that it in fact redirects so it will have two pages with the same content.
But if you revert, you will have the same problem, just the other way around, and you say that there are more pages indexed on domain without the hyphen than on the hyphenated domain.
The question about the sitemap from not2easy is a good one!
@sangi, it would be good to serve sitemap on the OLD hyphenated domain, with all URLs in there being WITHOUT the hyphen, and make sure the sitemap does not redirect. This will re-inforce the message to Google that the site has moved.
I would agree with you, aakk9999 if not the fact of newly reindexed urls on blue-widget.com with https:// (see OP). I was stunned after I discovered this!I missed that - you are right, this would indicate the issue could be domain-wide.
On the destination site, submit the two sitemaps you prepared previously containing the old and new URLs. This helps our crawlers discover the redirects from the old URLs to the new URLs, and facilitates the site move. [support.google.com...] .
Interesting thing: more than a year ago I've added the SSL to my new one bluewidgets.com. Everything is good, no traffic changes from Google, everybody's happy. One day I get the message from GSC: "SSL/TLS certificate does not include domain name https://blue-widgets.com/". "That's weird" - I thought, but without any research I just added the SSL to the old domain blue-widgets.com too.
As you see, it's a long story, but here's the final: even today if I do the "site:blue-widgets.com" Google search, I still get hundreds of thousands of indexed pages! What's even more weird, hovering on any of the links in SERPs I see the https version is indexed (say, https://blue-widgets.com/about-widgets etc.).
Just noticed one more funny thing in this G-riddle:
newly re-indexed old domain with https (https://blue-widget.com, which supposed to be gone from SERPs) has original (https://bluewidget.com) page titles(!). I include the domain name at the end of almost every page title (for ex.: "Page about blue widget - bluewidget.com"), so I'm 100% confident about this statement.
....we'd like to announce that we're adjusting our indexing system to look for more HTTPS pages. Specifically, we'll start crawling HTTPS equivalents of HTTP pages, even when the former are not linked to from any page. When two URLs from the same domain appear to have the same content but are served over different protocol schemes, we'll typically choose to index the HTTPS URL if:The above, as I understand it, is for discovery only... it's not about ranking, but it could be affecting what you see in site: search.
It doesn't contain insecure dependencies. It isn't blocked from crawling by robots.txt. It doesn't redirect users to or through an insecure HTTP page. It doesn't have a rel="canonical" link to the HTTP page. It doesn't contain a noindex robots meta tag. The sitemaps lists the HTTPS URL, or doesn't list the HTTP version of the URL The server has a valid TLS certificate.
Google's memory isn't just long, it's permanent. GWT still reports pages that haven't existed since 2006 as being suddenly found yesterday and 404 today. It happened so often in fact that I began suspecting someone was targeting my site maliciously but nope, just a long memory.
First... are you seeing these at all in competitive searches? ...or are they confined to site searches (which you'd mentioned) and perhaps also to searches for the old domain name?
If the latter... ie, if these are not for competitive searches... here's an old thread that discusses some possibilities relating to a change in what Google showed for domain-name queries....
Domain name replaced in SERPS with alias domain name
There was a follow up discussion here as well...
Google showing 301ed URLs in SERPs
I don't think that the above is the issue by itself, but combined with several other factors might provide a clue. Like your story, it gets complicated.
What I'm thinking... and I really haven't nailed this down, but I'm tossing it out as a possible clue... the inclusion of old domain in your SSL/TLS certificate might conceivably have triggered new Google indexing preferences to display https pages if it believes they exist.
My guess, and this is a guess, is that the quickest fix might be to include a "rel="canonical" link to the HTTP page", which is included in one of the conditions described in the list above. Other conditions might also apply.
Increase in “404” pages on http(s)://(www.)blue-widgets.com/
To: Webmaster of http(s)://(www.)blue-widgets.com/,
Googlebot identified a significant increase in the number of URLs on http(s)://(www.)blue-widgets.com/ that return a 404 (not found) error. This can be a sign of an outage or misconfiguration, which would be a bad user experience. This will result in Google dropping those URLs from the search results. If these URLs don't exist at all, no action is necessary.