Need Help with Google not Considering My Canonical Tag

[shaunm]

Hello All,

I'm having a weird problem with canonical. This is the first time I'm noticing Google not considering my canonical tag.

PageA is in English and has two more international versions too i.e German and French in local languages. PageA is old enough to be attracting a lot of quality backlinks pointing to it. At some point of time, PageA is discontinued because the product on PageA became extinct. But there are old customers who still wants to subscribe to that product so we have only removed it from the navigation and interlinking. As for Google, we put a canonical on PageA to PageB which sells the same product but in a different name and with a lot of new features, of course a new price tag too. But it's been months that Google is still showing up PageA for the product related keywords not PageB.

The only issue I see here is with the hreflang section. It could be giving a wrong signal to Google about which page to consider for indexing.

PageA (Eng)
Canonical = PageB(Eng)
Langue Alternate
DE = de/PageA
FR = fr/PageA
EN = /PageA

PageA (DE)
Canonical = PageB(DE)
Langue Alternate
FR = fr/PageA
EN = /PageA
DE = de/PageA

PageA (FR)
Canonical = PageB(FR)
Langue Alternate
EN = /PageA
DE = de/PageA
FR = fr/PageA

All these URLs' language alternates pointing to PageA as the ENGLISH alternative. But at the same time, the ENG alternative has a canonical to PageB. So, kind of a signal conflict here that Google is finding it hard to decide which page to index? I'm struggling to remove those language tags as the CMS setup needs some tweaks. What's your take on the issue?

Thanks!

[Andy Langton]

A canonical tag is a "hint" as to which page Google should use. Google will absolutely not follow your canonical instruction if the pages are different:

As for Google, we put a canonical on PageA to PageB which sells the same product but in a different name and with a lot of new features, of course a new price tag too

Hreflang is about telling Google the right language version to display and is not related to canonicalisation.

If your product has been replaced with a newer version, you should redirect it to the new version. You'll have a much higher success rate than with canonicals, but even then, if Google considers the pages to be different, you won't consolidate value either.

Otherwise, you should make sure that visitors to the old version are given strong direction to click through to the new one.

[aakk9999]

I have read somewhere (cannot find the reference right now) that the hreflang should point to a canonical version of URL. So it is possible that there are confusing signals sent to Google.

Ideally, you would implement 301 redirect as suggested by Andy above. But if you still want visitors who search for old URL to land on the old product page, then try removing hreflang from the old page and see if Google will start to follow your canonical directive.

[shaunm]

Google will absolutely not follow your canonical instruction if the pages are different:

I am sorry I don't think that's the case always.

Hreflang is about telling Google the right language version to display and is not related to canonicalisation.

It's about the Hreflang tag giving wrong hint to Google about which version to show when the browser language is 'ENG'. So, I am thinking it could also be the reason.

If your product has been replaced with a newer version, you should redirect it to the new version.

That's solve the problem. But as I said that page is still kept for old customers. May be we can do a redirect when the referral is Gooblebot, Bingbot or any other referring sites and for direct visits, or known IPs we can remove the 301 redirect. Not sure how feasible it would be though.

[shaunm]

I have read somewhere (cannot find the reference right now) that the hreflang should point to a canonical version of URL. So it is possible that there are confusing signals sent to Google.

That's what I suppose too.

But if you still want visitors who search for old URL to land on the old product page, then try removing hreflang from the old page and see if Google will start to follow your canonical directive.

My dev team is having a hard with removing such tags which are dynamically added by the CMS. Fingers crossed.

[aakk9999]

May be we can do a redirect when the referral is Gooblebot, Bingbot or any other referring sites and for direct visits, or known IPs we can remove the 301 redirect. Not sure how feasible it would be though.

I would not do that - you could be penalised for cloaking.

If your dev team has difficulty in removing dynamically created hreflang, then the alternative is to leave hreflang on the PageA, but to make sure that hreflang points to a canonical version of the page (i.e. to point to language version of PageB).

[shaunm]

I would not do that - you could be penalised for cloaking

Isn't cloaking all about giving users a different version than search engines UPON CLICKING a link in SERP? If it's Googlebot, then it will continue to see the redirect and visitors clicking from SERP too (assuming the redirected page is still in index). But if it's a direct visit, email clicks where the referrer is not Google.com* then the redirect won't be there. Am I talking about something which is technically possible and is still dangerous?

If your dev team has difficulty in removing dynamically created hreflang, then the alternative is to leave hreflang on the PageA, but to make sure that hreflang points to a canonical version of the page (i.e. to point to language version of PageB).

That sounds like a good idea. I should check if they are at least about to modify what's in there already.

As I was typing this reply it also came to my mind 'why not just use noindex robots on PageA instead of beating around the bush?'. Is that a valid question in my situation?

Thanks!

[aakk9999]

Isn't cloaking all about giving users a different version than search engines UPON CLICKING a link in SERP?

Cloaking is showing Googlebot a different content than to a visitor when Googlebot requests the page [support.google.com...] (btw, Googlebot does not follow clicks).

why not just use noindex robots on PageA

Yes, this could be another way of doing it, but then you should remove rel=canonical from the page that is noindexed, it is not a good idea to use them together - Google recommends that you do not use noindex and canonical on the same page.
John Mueller from Google on noindex and canonical together
http://www.seroundtable.com/noindex-canonical-google-18274.html [seroundtable.com]

However, your primary aim is:
- Google ranks your pageB in SERPs
- Direct hits to pageA still end on PageA
- pageB gets all credits/juices that currently point to pageA

In which case - use of canonical and pointing hreflang to the canonical version of language URL is probably the best bet to force Google to update its index.

[shaunm]

I'm a bit confused.

Scenario 1:
The server identifies a crawler through user-agent to be Google bot so the server responds with a 301 for PageA. Now, the PageA is probably replaced with PageB in Google index. Even if PageA is not replaced in Google index for sometimes, the visitors clicking on PageA from Google SERP will still be redirected to PageB. So, I'm not actually showing two different versions to Google and the users who is landing on my page from Google.

Scenario 2:
PageA is typed directly into the address bar OR it's clicked from an App, software/tool which opens up PageA in a browser(whose user-agent is excluded from redirection). Then the server will treat PageA as 200 ok page and servers the old content. Does that make sense at all?

[aakk9999]

So, I'm not actually showing two different versions to Google and the users who is landing on my page from Google.

The issue is in the bolded text above (emphasis mine) which should not be in the above statement.

Cloaking is showing a different page to the visitor than what you show to googlebot fullstop. It does not mean that the visitor has to arrive from click from SERPs.

If Googlebot requests PageA, your server will respond with 301 redirect to PageB.
If a visitor requests PageA, your server will respond with 200OK abd it will serve PageA.

So Googlebot and Visitor do not get the same response nor the same page from your server when a request is done for PageA and therefore you have an issue of cloaking.