Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Download Pages of Our Products are non-indexed by Google Suddenly

         

vicky2885

11:39 am on Sep 17, 2018 (gmt 0)

5+ Year Member



Hi, i have a question about pages indexed, hope you can help me out, thank you in advance.

Last week, we found our French site and Arabic site's download pages were not indexed by Google,

The url of French site is: www.example.fr, the two non-indexed pages were: https://www.example.fr/example2/download.htm and https://www.example.fr/rescue/download.htm

Then we made some changes on last Friday, like removed JS redirect from download page and removed meta no index from the download pages from mobile versions (Because our products are desktop software, for mobile users, we designed some pages to convert mobile users, only work for mobile users. Normally, we add “no index” to block them from Google crawling). Then the two sites' download pages were indexed by Google on Saturday and Sunday.

However, when we get office today, only to find the Japanese and German site's download pages are not indexed by Google.

The url of German site is: www.example.com/de/, the non-indexed page is: https://www.example.com/de/rescue/download.htm
For Japanese site, the site is: www.example.jp, the non-indexed page is: https://www.example.jp/rescue/download.htm

Our website is not a new site, and the two pages always rank well in Google for years, because the organic keywords are brand related, like Example Download or Example2 Download, or just search Example or Download. Recently, we found the two pages' organic traffic declined sharply, and then found they are disappeared from google SERP, and they are not indexed by google.

To fix this problem, we had tried the following methods:

1. Checked the two pages have “no index” tag in the page source – No
2. Checked robots file and see if we blocked the two pages from crawling - No
3. Checked the two pages’s re=canonical tag and see if they are linked to the wrong pages – No
4. Checked if the pages for mobile phones has “no index” tag (Because our products are desktop software, for mobile users, we designed some pages to convert mobile users, only work for mobile users. Normally, we add “no index” to block them from Google crawling) - Yes, they have.
5. Checked “Remove URLS” to see if we removed the two pages from google SERP by mistake – No.
6. Submit the two pages to Google via URL Inspection, but they are still no indexed.

I want to know why our pages are disappeared from Google's SERPs, and not indexed, and how to make them indexed again. If possible, also hope you can check if the two pages have any not-friendly codes in page source.

Than you very much!

[edited by: goodroi at 1:18 pm (utc) on Sep 17, 2018]
[edit reason] Welcome to WebmasterWorld, now please go read the forum rules :) [/edit]

goodroi

8:25 pm on Sep 17, 2018 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



To clarify your situation, how did you test to see if the pages were indexed in Google? Did you look it up in Search Console? Did check for a specific string of text that is only on that specific page? Did you do a site: search? Did you just see no referrals from Google?

Do all of these pages have a significant amount of unique text? Do all of these pages have backlinks pointing to them?

Have you checked to see if there might be duplicate content on another website or a possible outside hijacking attempt? I doubt this is the case but want to be thorough.

Have you checked the status codes in your log file the last time Googlebot visited those pages? Do you have a server monitoring service so we can rule out that your server might have been down or timed out when Googlebot visited?

In the meantime, I suggest adding some unique text to these pages that is regularly changing. Google tends to more frequently visit pages that are updating more frequently. Just make sure it is real content. Google doesn't respond as well to randomly generated gibberish.

keyplyr

12:21 am on Sep 18, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hi vicky2885 and welcome to WebmasterWorld [webmasterworld.com]

Sometimes Google will remove a page if the topic/theme is very close to another page on the web site and that other page has more/better topic related content.

vicky2885

4:00 am on Sep 18, 2018 (gmt 0)

5+ Year Member



Hi, Goodroi!
Thank you for your reply. About the question you mentioned, i will answer below to help you know our situation. Because we are stuck on this problem, we do need the help from you, thank you again.

1.how did you test to see if the pages were indexed in Google?
We search the brand + download on google, or use site:example.com/rescue/ download , but with the two methods, we can ‘t find the target pages.

2.Did you look it up in Search Console?
We searched this page in URL Inspection, and it does show the two pages are not indexed by Google.

Because i can't add screenshots here, i extracted some useful information here:
For German site: it shows:
1) Coverage: Duplicate, submitted URL not selected as canonical.
2) Discovery:
sitemaps: [url]https://www.example.com/sitemap.htm[/url]
3) Crawl:
Last Crawl: Sep 17, 2018, 11:37:05 AM (Because we tried to submit this page to google with this URL Inspection tool)
Crawl allowed: YES
Page fetch: Successful

4)Indexing:
Indexing allowed: Yes
User-declared canonical: [url] https://www.example.com/de/rescue/download.htm[/url]
Google-selected canonical:[url] https://www.example.com/de/resuce/download-android.htm[/url] (For this, i will go to this page, and change the rel=canonical and link to download.htm, is this ok? )

For Japanese Site: URL is not on Google
1) Coverage: Crawled - currently not indexed
2) Discovery:
sitemaps: [url]https://www.example.jp/sitemap.htm[/url]
3) Crawl:
Last Crawl: Sep 11, 2018, 10:21:23 AM
Crawl allowed: YES
Page fetch: Successful

4)Indexing:
Indexing allowed: Yes
User-declared canonical: [url]https://www.example.jp/rescue/download.htm[/url]
Google-selected canonical: Inspect URL

3. Did you do a site: search?

Just like I answered in Question 1, we did search them with site, and found they are not indexed by Google.

4.Do all of these pages have a significant amount of unique text? Have you checked to see if there might be duplicate content on another website or a possible outside hijacking attempt?

For this question, our German pages’ content are unique, we can’t find any similar content on other pages. While Japanese content can be found in others sites, because it only mentions this version supports the operating systems, and iPhone models. But I don’t think this will become a reason makes both of them disappeared from Google.

5.Do all of these pages have backlinks pointing to them?

Yes, both of the two pages have many backlinks from high-authority sites. And the numbers are growing.

6.Have you checked the status codes in your log file the last time Googlebot visited those pages?

We used the Fetch as Google in Google Search Console to check the two pages, both of them show the status as “Complete”, and URL Inspection also show both of them can be crawled, but not indexed.

7.Do you have a server monitoring service so we can rule out that your server might have been down or timed out when Googlebot visited?

For this, I don’t think it would be this reason, because we just found the two pages are not indexed, and others pages can be indexed, and get organic sessions from Google.

Thank you very much. Today, we also try to add some inner links to the download pages from some pages with high-authority, and check if the two pages will be indexed.

[edited by: phranque at 6:39 am (utc) on Sep 18, 2018]
[edit reason] exemplified urls [/edit]

vicky2885

4:11 am on Sep 18, 2018 (gmt 0)

5+ Year Member



Hi, keyplyr!

Thank you for your reply. I often visit this forum without registration, just registered an account yesterday and asked this question.

About what you mentioned, " Sometimes Google will remove a page if the topic/theme is very close to another page on the web site and that other page has more/better topic related content", according the information URL Inspection shows (just i mentioned in last post), i think it would be this reason for German site, because it shows Google picks another page like exmaple.com/de/rescue/download-android.htm as canonical to the download page. I will try to add a rel=canonical and link to download page on download-android page, and see if the german's download page will be indexed.

keyplyr

4:51 am on Sep 18, 2018 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



When building new pages, I always try to keep each unique and not compete with other page's indexing. Difficult to do when the pages are of the same basic topic.

Canonical tags have their use, but sometines they get in the way.