New Content Indexing issues

[Jori]

Starting somewhere this year, all of my new content is not beeing indexed.

I wrote 4 articles : 2 in january, and 2 in february.

All contents are written with my two hands, long and extensive articles about one topic.
No technical issue, good performances with pagespeed.
Internally linked, one from the homepage.

If I update an old article, no problem, it will be reindexed.
It's just my new articles.

In other websites I have, my new articles are indexed in hours.

We are numerous reporting this (specially in my home country, France).

[robzilla]

Did you run it through the URL Inspection tool in Search Console? You can also request indexing there.

Do other search engines index these pages?

[Jori]

No issues on bing or ddg, and yes, of course, i run it in the inspection tool of the GSC, asked for an indexation etc.

No love from G.

[NickMNS]

How did you test for indexation?

[Jori]

with the command site:
but even without that, you can see it on the search console, with a "explored, not indexed" status.

[NickMNS]

The "site:" command is not reliable and GSC may be showing you false information.

The most reliable method to check if a page is indexed is to do search for a relatively long block of unique text from the page in quote, for example do a search for the entire first paragraph in quotes. If the pages has been indexed Google should return the result. You can also try it with pages that you know are indexed just to be sure that those show up.

[Jori]

Yes I also did the paragraph search in quotes. I know my job ^^ .. Google simply doesn't want to index it. For no particular reason.
The only reason i can see : i'm filtered.

I don't have manual penalties.

[robzilla]

Well, you're fun to help. If you know your job, what's your question for us? I'm sure you've seen the checklists for the "Crawled - Currently Not Indexed" issue. If nothing applies, you'll have to dig a little deeper. For inspiration: A Curious Case of Canonicalization [gentofsearch.com]

[Jori]

Because I have no clue what's happening... :'( i' ve done all the essential things, maybe someone had the same issues?

I'm looking for some weird filter I'm not ware of. I'll check your link, thank you!

[Jori]

Ok, I checked the link for inspiration you gave me, robzilla.
It's a very interesting read, but... I removed the feed(s) years ago. If someone try even to access it by typing the classic URL for the feed of a page, he will get a error 500.

I'll have to keep searching, but really, I'm clueless.

[frankleeceo]

Basically Google doesn't think it's unique enough to warrant indexation. I see it with some of my sites.

Chances are. if you somehow get these pages to index & rank. You end up eating your past traffic.

"All contents are written with my two hands, long and extensive articles about one topic."
Long & extensive articles about one topic lead to pretty straight forward cannabilization. Because you are just talking about the same thing in different ways, resulting in similar keywords / ideas between all the different articles. At the end, only "supposedly" the best version ranks and gets all the love. Leaving everything else to zero.

[Jori]

I thougt about that, but i wrongly express myself. Each article talks about one topic, different from others. Ok, within the global main subject. I don't think it's a quality issue of the article, it's a global issue of my website.

No new content for me.

Google thinks that my website is rubbish. It didn't think so back then in 2008 when I created it.

Maybe my links ? I'm still heavily linked by numerous spam plateforms.

Sooo... if you're clean, don't try to manipulate links and try to compensate by writing even more "good" articles, what's the reward? New articles not even indexed !

[frankleeceo]

The way to approach content generation is to understand your overall site traffic. New content does not necessarily generate traffic to those specific pages, but sometimes it boosts relevancy signals for other content from the site.

Make new content when they benefit your users somehow.

At least how I think about content has changed over time over the years.

I don't make new content to get more pageviews to those pages. I make new content to stay as a de facto leader in my niche.

[robzilla]

It's a very interesting read, but... I removed the feed(s) years ago. If someone try even to access it by typing the classic URL for the feed of a page, he will get a error 500.

Mostly it's an example of how Google can sometimes get canonicalization wrong. So it's not strictly about the feeds, but the possibility of things being misinterpreted; if you've covered all other bases you may have to think outside the box at these sorts of things.

No new content for me.

Google thinks that my website is rubbish. It didn't think so back then in 2008 when I created it.

When was the last time you published a new page on this site that was indexed?

[Jori]

It was in November. In a French seo news website, others have reported the beginning of indexing issues also in November. [abondance.com...]

[Sissi]

Jori

I do French SEO as well
Noticed that G as soon as it discovers a similar content it leaves it now as quasi duplicated content
G is just filtering any new content without added value apparently

[Jori]

What is "added value" in 2022..

[Sgt_Kickaxe]

Just an observation, I've seen this on pages that are #1 worthy. Sites with never changing index pages and/or category pages showing too many article links tend to have these more often..

Add a mix of recent and best content internal links to the index page and this tends to stop. Reducing the number of links per category page to 10-20 with most recent appearing first seems to help too.

Google checks these pages more frequently so it helps if they find links to your newest and best stuff when they do. It helps users too.

[edited by: Sgt_Kickaxe at 10:55 pm (utc) on Mar 23, 2022]

[Jori]

This is very good advice I think. I suspected something like that also, and I linked one of my non indexed articles directly in the home page recently. With no luck for now. I'll dig deeper in that direction.

But still : it's curious when you think about it. My website is not about "recent" or "old" articles, I only work on cold content, put in a special place in the structure of the website.
Maybe, juste maybe, Google is not thinking the same thing than I.

Let's imagine something. Google have a way to classify websites. Type A, B, C...

If for Google, my website is a "type A", it demands that my website looks like a "type A" and will be indexed accordingly.
If my new content doesn't correspond to the type A, it will not be fully indexed. I have to wait for Google to reassess the type of my website.

This is plain guessing, but I have to get out of the box :)

[Sgt_Kickaxe]

Google determines which your most important pages are and checks them most often. Your internal link structure plays a role in that. It helps if they link to new content and, occasionally, old content that got no external love to prop it up. Too many internal links on a page, however, dilute the downstream benefit. Maximize the value of your index and category pages without going overboard and this isn't usually a problem, for long anyway.

I think it's in Google's guideline manual somewhere, it has been for a long time. I think Matt Cutts once did a Q&A about the importance of Google easily figuring out what's new and what's best on your site.

Sadly those old Matt videos seem to be getting replaced with updated versions in Youtube's Google Search Central channel. "They was good stuff". If you don't know who Matt was that's OK... and I suddenly feel older saying that, I remember when I could set the Google crawl rate myself in my "webmaster tools" oi.

[Jori]

I did my first website in 1998, so yes, I really know who is Matt Cuts. We used to trash him back in the days... and now we are crying for him to come back ^^"

You're talking about "link juice" and "link distribution"? I'm always thinking and doubting about it (am I doing it right?), and I try to take it carefully : where do I link my new content ? It's a decision that I always took based on data...

I did a change in october that I didn't mention before : I remove all the structure of the website in the urls. No more example.com/category/subcategory/article , it's directly example.com/article

I was doing it bit by bit during all of 2021, and my testings were looking good. So I did it for all of the website in october, but with a good breadcrumb.
It has hit me hard because of all the redirections, but new articles were correctly indexed, until end of november.

[NickMNS]

@Jori,
Off topic...
the link you posted was great, I'm from Quebec and there is a real aversion towards using English terms in French. My favorite is:

depuis un site « trusté » par Google

Back on topic. But there is your answer, it's not just you, and there seems to be an increased need for quality. While your pages may not be "low" quality they are being categorized as such by Google. Have you checked your link profile for an excessive amount spam links pointing to your site, or scrappers or hot-linkers stealing your content in such a way that could reflect poorly on your website. Basically is there anything that make Google question the quality of the website?

[lucy24]

<tangent>

If someone try even to access it by typing the classic URL for the feed of a page, he will get a error 500

Uh .... Why? Nothing should ever result in a 500; it's an error in the deepest and truest sense of the word.
</tangent>

[Sissi]

Jori

added value is for example talking not about Sushi but probably about something more exotic
To the solution of your problem: I was following one at webrankinfo and it worked

[Jori]

@NickMNS : in France, we don't feel much about beeing "invaded" by US (but we should...) ^^
Yes, you're right, I'm heavily beeing attacked by nseo for a long time know. I disavowed more than a 1000 links, utter rubbish links. They are so blattantly lame that I want to slap in the face of John Mueller each time he says that they "can manage it very well".

@lucy24 : the feed urls are not beeing crawled. The 500 error it's there to prevent any crawlers to come back, but maybe I can be wrong.

@Sissi : where is the advice in WRI? thx

[NickMNS]

The 500 error it's there to prevent any crawlers to come back

A 500 error does not achieve that goal. Googlebot is likely to check the link again intermittently to see if the error has been resolved. If you want to send the signal that the page is gone the URL should return a 410 response.
[developer.mozilla.org...]

Yes, you're right, I'm heavily beeing attacked by nseo for a long time

Disavowing links doesn't seem to do much if anything at all. If you haven't done so already, you need to try to block the scraping. The best way I have found are, prevent other websites from framing your content, block hot-linking of images, and block bots from accessing your site (easy to say harder to do).

[lucy24]

Googlebot is likely to check the link again intermittently to see if the error has been resolved.

Yeah, that's what worried me. Even if there are no human-visible links to the old URLs, G (and, even more so, bing) will continue requesting them periodically, from now until the heat-death of the universe. And if they consistently get a Server Error response, it might make them look a little askance at the site-as-a-whole.

[Jori]

I changed 500 to 410, let's see what happens. But in GSC, I didn't get any errors at all, not even in my apache logs last time I checked.

The kind of nseo I get it's the kind where you can not do anything I guess. One of the many techniques they used is to point in 302 a heavily linked scam website to mine. Really simple to create utter rubbish, and use of networks of free blogs also, etc, etc.

The only way I was thinking to prevent this, is to acquire legit links. If I play by the book, maybe in 2047 I will get back on tracks. Others don't hesitate to create heavy nseo campains...

[PawelPolska]

Hello,

Translated using an interpreter: I'm sorry for that.

From around March 19, I have a similar problem. This applies to one domain. I used to get good results in the search engine by then. The main phrases quickly reached TOP 1-3 and 3-5. The published pages were visible from a few hours to a maximum of 1 day.

About March 19, new entries do not appear in Google results. The remaining subpages have consistently high positions. However, no new ones are published in the search engine.

They are visible after entering "site:siteaddress.com" and setting the time to e.g. 24 hours or the last week.

They are also visible in the browser after entering a keyword and setting the publication time to the last 24 hours.

In GSC, the page is "green" with the note that the address is on Google. "Page submitted and indexed".

The subpages have graphics inside. Google Graphics also indexes them after entering "site:siteaddress.com", but in the "main phrase" they are completely invisible. As in the case of subpages, graphics were displayed very quickly in general search results before.

I try different solutions but so far no results.

Greetings,
Paul

[Jori]

I got an external link pointing to one of my non indexed pages : it solved the issue, for that page, and that page only.