Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Weird Indexing Issues since late October (anyone else?)

         

Kratos

3:31 pm on Nov 30, 2021 (gmt 0)

10+ Year Member Top Contributors Of The Month



I'm having some very weird indexing issues since late October.

Basically all new articles since late October are appearing as "Discovered - Not indexed" and only some of the image attachment pages of the articles are being indexed while the main article itself is not.

Actually, only one published article since late October has been fully indexed and it took less than 12 hours to index, which has been the usual with this website.

The website is a reference in its industry and the articles are all in depth with original pics. Articles are all original and written by an accomplished expert in the field and are over 6,000 words long with anywhere from 30 to 50 original images.

The website is white hat and it has continued to gain traffic from Google including in this last update. The website is a high quality website and is referenced by its industry's figures.

I'm not sure what's going on since articles have always taken less than 12 hours to get indexed in all these years.

I've read of some indexing issues that Google experienced in November, so not sure if this is related to it. Surely Google should have solved any indexing issues that were reported.

I've also read that Google sometimes delays indexing when going through a big update like this November one.

But it's weird seeing all new articles (except one) still showing as "Discovered - Not indexed" and not being indexed on Google search (while some of the articles' attachment image pages are indexed?!).

Even worse is how one of the articles appears as indexed on GSC but, when I do a site search, it is NOT indexed.

This is crazy.

Things I've checked:

1) Robots.txt and meta tags. All good and Googlebot can crawl the articles and see them.

2) No errors are thrown when I request the URLs of the articles on GSC and when I Live Test them. However, it says N/A for "Last crawl update" on all articles that are "Discovered - Not indexed" which is crazy since many of these articles are 4 weeks old and I've seen them being crawled by Googlebot on my traffic logs.

3) All articles that are "Discovered - Not indexed" are linked from the homepage. The homepage appears as crawled and indexed every day, so obviously Googlebot knows that the articles are there!

4) I've resubmitted sitemaps several times every week. Nothing. I've also requested the articles to be submitted on GSC repeatedly. Still nothing.

5) Articles have NOT been scraped by spammer websites. When I do a Google search for sentences and paragraphs, nothing comes up.

6) When I Google search the snippet of these articles that appear on the homepage (the homepage is crawled and indexed every day), nothing comes up either!

7) All articles ONLY appear on the website in full in their corresponding URLs. They only appear with a small text snippet in the homepage and in archived pages (e.g. categories, tags, etc). So no duplicate issues either (and all articles have their correct canonical listed).

8) All articles that are not indexed on Google and which appear as "Discovered - Not indexed" on GSC do appear indexed on Bing and come up when I do a Bing search.

Has anyone experienced similar weird indexing issues since October or November?

Do you guys have any ideas on what could be going on given how this website has always had its articles indexed in less than 12 hours for many years?

Thanks!

aristotle

7:39 pm on Nov 30, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Have you tried typing in the exact URL's of these articles into google search? This might be another way to get googlebot to crawl the pages and perhaps get google's algorithm to give them more consideration.

poweri

9:19 pm on Nov 30, 2021 (gmt 0)

10+ Year Member



I have the same problem since mid November. New websites are no longer indexed. The last successful indexation was on 16 November.

From past experience, I know that indexing is difficult during a major update. Sites submitted during a major update either became unindexed or at least a month later.

Kratos

11:09 pm on Nov 30, 2021 (gmt 0)

10+ Year Member Top Contributors Of The Month



Have you tried typing in the exact URL's of these articles into google search? This might be another way to get googlebot to crawl the pages and perhaps get google's algorithm to give them more consideration.


I've run a site: (command) with the URL for all articles on Google search, which returns no results (i.e. articles not indexed). I wasn't aware of this trick to try to get Google to index the articles.

Thanks for the tip, Will try it out.

I have the same problem since mid November. New websites are no longer indexed. The last successful indexation was on 16 November.

From past experience, I know that indexing is difficult during a major update. Sites submitted during a major update either became unindexed or at least a month later.


I could understand this for any new websites, but this is a white hat website with references from industry governing bodies and it has never had any indexing issues ever. Up until late October, articles would index in less than 12 hours without fail.

The website is also years old and very well established. It even has benefited from this last November update, so it blows my mind how Google is failing to index the new articles since late October.

According to Google's Webmaster help page (which offers practically zero help), a message of "Discovered - Not indexed" means that Google is aware of the URL but hasn't yet indexed it because the "website's server may be overloaded when the crawler visits the page" (I'm paraphrasing, but it's something along those lines).

The website runs absolutely fine and doing a Live Test on all articles shows ZERO errors and that the articles are available to Google.

It's been over 4 weeks for some of these articles since they were published and are still not indexed and showing as "Discovered - Not indexed" on GSC.

I'm lost as to what the hell is causing this lack of indexation of all new articles and labeling as "Discovered - Not indexed".

GSC is also reporting a few articles indexed, yet they do not appear on Google search results when running a site command or when typing sentences with "" on Google.

Thanks so far for the replies. Any others suffering from indexing issues as of the last 4 to 6 weeks?