October 2023 Google Search Observations

[Cyril TechWebsites]

---

System: The following 3 messages were cut out of thread at: https://www.webmasterworld.com/google/5093320.htm [webmasterworld.com] by not2easy - 11:10 am on Oct 1, 2023 (atl -4)

---

Seeing further drastical drop on Saturday. I stopped producing new content and updating the old content. There is no sense anymore for me, feeling awful to say that, but my website was fully destroyed with the "helpful" update.

[RedBar]

What are the correct robots.txt to block AI?

User-agent: chatgpt Disallow

Which others ?

[not2easy]

There are scattered mentions around here, one recent was for Bard and Vertex AI LLMs: [webmasterworld.com...]

[ichthyous]

@RedBar I have these...the first six are AI scraper bots. After that are listed the older web scrapers:

User-agent: CCBot
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: Claude-Web
Disallow: /
User-agent: Httrack
Disallow: /
User-agent: HTTrack
Disallow: /
User-agent: Wget
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: SeznamBot
Disallow: /
User-agent: DotBot
Disallow: /
User-agent: BLEXBot
Disallow: /

[not2easy]

Bing's how-to: [webmasterworld.com...]

That is to cover Bing Chat and training Microsoft’s Generative AI.

[l1fejosh]

Down 37% in organic traffic after a week of amazing conversions, dropped like a stone on Friday-Saturday: -13%.

[renatovieira]

Today the drop will be huge... Here we go again...

[RedBar]

@ichthyous

Thanks, I had the first 4 and seen some others mentioned elsewhere.

[Soulofmonk]

Google is dead set to kill my site this year. :(

[christianz]

According to SEMrush, I'm still losing far more keywords per day than gaining. This has been going on since September 18th. Even after the update finished, I've lost some #1 rankings to high-authority sites.

Google recommends that we should fix or remove unhelpful content. So should we update all posts that got affected? Is anyone here updating their posts that got spanked by Google?

Same dynamic. Drop/slide since 18th.

Unless you deliberately added lots of spammy content, you don't have to change anything. Their "classifier" is clearly faulty.

All you can do is (for basic hygiene) block ChatGPT and Bard in robots.txt. Failing to do so will be interpreted as green light to grab, spin and monetize your content however they see fit, without any compensation and legal recourse.

[RedBar]

Global site for Saturday and Sunday both at 62%, the previous weekend 63.5% and 61.5% !

I'm tendng towards my global sites of being a company brochure, it's been done, walk away.

Meanwhile our hotel venue, a local site, had its second highest 2023 PVs yesterday and, pleasingly, a fantastic till take.

[ichthyous]

Today the drop will be huge... Here we go again...

My site is up strongly today, after 10+ days of lower traffic and ZERO conversions for 8 days in a row...not a single inquiry or even a phone call. Just students asking for internships or jobs. Things are very unstable right now

[Helix Online]

@ichthyous My certain articles ranked higher than the authority sites before this update. After all, site authority is not the only ranking factor. This update was supposed to be about promoting helpful content, not promoting site authority. Google's advice to update or delete "unhelpful content" is a bad idea, then. Why didn't they just name it "authority update"?

There's not much to do to increase site authority except to keep publishing "helpful content" and wait for the backlinks to come in. Link building is too risky.

[christianz]

There's not much to do to increase site authority except to keep publishing "helpful content" and wait for the backlinks to come in. Link building is too risky.

I have plenty of organic backlinks and zero paid backlinks. Often more than the so called "authority" sites. It doesn't help. If I had a magic wand and could turn into a billion dollar media company or US government .gov site, that would help. So I need to work on that and forget about my actual content.

[ichthyous]

My certain articles ranked higher than the authority sites before this update. After all, site authority is not the only ranking factor. This update was supposed to be about promoting helpful content, not promoting site authority. Google's advice to update or delete "unhelpful content" is a bad idea, then. Why didn't they just name it "authority update"? There's not much to do to increase site authority except to keep publishing "helpful content" and wait for the backlinks to come in. Link building is too risky.

Domain Authority has been an important element in ranking for as long as I can remember, we are all subject to that. The problem with a lot of articles is that competing content is immediately being dumped online by large legacy media sites like Newsweek that have discovered that they can bring in ad revenue by posting articles 24/7 on every topic under the sun and capitalize on their high domain authority.

That makes it very difficult to compete unless 1) you write about increasingly more specific micro-topics, which means a very narrow audience, or 2) you build your own site into a brand of some sort and people have brand recognition and go to your site directly. Lots of luck with that, it's an almost impossible task and I think a lot of bloggers who built high traffic blogs are going bye-bye unless they also have a huge social media following to link to their articles. Now with AI it's about to get a lot worse for all of us I think...you can publish well written junk articles at the speed of light.

[christianz]

The problem with a lot of articles is that competing content is immediately being dumped online by large legacy media sites like Newsweek that have discovered that they can bring in ad revenue by posting articles 24/7 on every topic under the sun and capitalize on their high domain authority

And that is the true "SEO first" content that should have been targeted with a "SEO first" content penalizing update.

Also, traditionally domain authority was derived from backlinks (before they become 100% spam links). Which meant it was competition based on merit (content quality). After these recent updates domain authority is purely off-site factors that have nothing to do with content quality.

[EditorialGuy]

Also, traditionally domain authority was derived from backlinks (before they become 100% spam links). Which meant it was competition based on merit (content quality). After these recent updates domain authority is purely off-site factors that have nothing to do with content quality.

Some of those off-site factors may have a lot to do with content quality. As with most things, it depends.

As for the heyday of PageRank (a.k.a. backlinks as "votes"), we've come a long way from the original concept of the World Wide Web. How many links these days are true citations or references, as opposed to a form of SEO currency?

[Sodero18]

Since September, I've been checking search results on Google, Bing and DDG and the difference, as other users have mentioned, its significant. Results are more relevant and better quality on the other search engines and I'm assuming users are starting to understand that.

I really feel for all the website owners and developers who have lost 50% or more of their traffic. If this is affecting millions of sites, the economic impact is staggering.

[christianz]

Some of those off-site factors may have a lot to do with content quality.

May or may not. There is too much collateral "baby with the bathwater" damage if you only rank sites with good off-site factors. Many types of websites don't need off-site profile, they can exist purely in the web and anonymously.

How many links these days are true citations or references, as opposed to a form of SEO currency?

Probably one in 5000 - 10000 is real, looking at ahrefs. And it continues to get worse. I am not saying links exclusively should be used like 20 years ago. I don't know what is the solution.

[swright]

I'm seeing actual helpful pages demoted and outranked by sites that copied their content. In one particular egregious case, a page on an old established site was demoted from top 3 to the 2nd page and half the sites that Google promoted ahead of it actually copied the content from it and rephrased it slightly. Basically G has promoted worse copies of the original to the top and demoted the better original. The promoted sites don't seem to have better authority either. This is not an isolated case - I've seen many such examples over the past week or so. Helpful content update - my a$$. This update has nothing to do with helpfulness from what I've seen so far. It doesn't make sense at all.

[BlueEyes82]

The Google quarterly figures seem to have been reached, there is a medium “rollback” in my area. My sites that lost 60% have increased by 20-25% again, and the situation is similar for my competitors who have been penalized.

Even sites that were hardly affected by the September update are showing more movement.

[mosxu]

Huge increase in zombie traffic yesterday really need to be on top of it as it spends all campaign budgets!

[Micha]

The Google quarterly figures seem to have been reached, there is a medium “rollback” in my area.

I'm observing similar things in my thematic area, however, it's more than a recovery for me. In some cases, I see an improvement of almost 80 percent and Google News traffic has almost tripled since the rather lousy weekend. Also, it seems that the big news sites are losing.

[Razorllama]

Huge increase in zombie traffic yesterday

I'm seeing the same here, conversions over the past 3 days have been 30% of what they usually are. Even after the traffic drop, I compared current traffic levels with similar traffic levels and the traffic quality is just abysmal.

[RedBar]

Bearing in mind the majority of my business is in Eurasia, Indonesia & Middle East, I usually have good overnight traffic however this morning after 10 hours of my Googleday I'm at 12% and those have been solely single page views.

More of a mess being churned out?

[Maximiliam]

Been a helpful and totally free site for over 20 years with answering peoples car repair questions (writing staff have been actual professional car mechanics). We let people read the solutions to others peoples problems in a moderated forum style environment. We even have large archive with articles (step by step mostly) and video guides on YT.

We got hit massively by this update. We are down at least 50% and it just gets worse. Even though we are the definition of being helpful and have a legacy site. I guess we have not been helpful all this time and I guess our several hundred thousand users have disliked us all this time after many of them praising the good work we have provided for them.

We have done nothing wrong to our knowledge. At least not per earlier guidlines from both the community and Google.

Google does not seem to understand our subject at all (car repair). There is not a single solution to a problem. There might be several and it might take some going back and forth coming to a conclusion. Sorry that we helped so many people. But clearly Google thinks we did not.

[christianz]

@Maximiliam

Very similar situation, although my drop/slide hasn't (YET) been as dramatic.

Being car repair site I think your queries are mostly long tail and very specific. This is where Google is most broken. For long tail queries they routinely show hacked sites on random subdomains with generated content.

They also dumb down your queries. If you search for replacing "camshaft sensor", they will give you results for replacing "crankshaft sensor" and even boldface the non-existent, incorrect keyword "crankshaft" which you didn't enter.

[Fluff_Nutz]

Funny. Is it all down to consistency? During the update my team were posting articles every day. Granted the traffic dropped a good 45-50% still but once it ''ended'' I got the traffic back. However, as I cannot afford to play Google's games right now I took one of my writers to another platform. Where consistent growth and traffic can be found. Meaning the site is now on hold and no new articles have been posted for several days now. Looking at my traffic and it is beginning to drop. As expect with negligence.

I don't notice too much of an increase with just one or two articles. But posting roughly 5 or more per day seems to help. With AI you can post hundreds a day. Meaning this might be why they are gaining top positions. It's all down to posting quantities of articles per day. G's way of determining which sites are actually 'in their eyes' more into their business, should we say.

So perhaps, in theory, by stopping posting. Your positioning is decreasing for those that post daily quantities. Most sites at the top do exactly this. It's stupid though. How can quality be found in articles that come in quantities each day. Your going to either burn out or over time not put in as much effort. Sites should not be valued like this. But how else can they be looked at? Without sending an expert. Hence site dominant authorities. Which again, can be rigged.

The whole internet is just corrupt and dirty. I keep saying that I'll return to the site one day but, really, is it worth it? G's logic is terrible. The wrong sites are constantly clogging the top positions due to terrible algos or deals. It's all wrong.

To further confirm my theory. I remember, a while back, reading on Reddit that someone managed to reach the $2,000 mark using AI. All within one single year. They said what helped them the most was posting quantities of articles per day. AI sites are having a field day right now. But AI should not be the answer. They cannot be relied on.

So my bet is to either join the AI or make enough money elsewhere to pay for quantities of articles per day. You can always write them yourself but, you will likely burn out. As I learnt last year. Hence why I ended up hiring a team. Long story short G's logic is quantity over quality.

For those seeking further proof on why Google is a terrible search. I remember looking at Bing. One of my articles was placed before a site with higher dominance. One that had been around longer than my site. I, in G's eyes, should never be ranked above them. But on Bing I was. Why? Because my article featured videos and pictures. Whilst this domain site had neither. That is how sites should be ranked. Both images and videos further help the viewer. Giving a better form in 'helpful content'. Clearly G has no idea what this word even means

[christianz]

@Fluff_Nutz

I would think that large number of articles/day for a small publisher should indicate spam if nothing else.

I would only publish AI articles on a throwaway site, not a site I care about or one that has any loyal following. Nobody wants to read any AI content. There are already more AI generated articles than 7 billion people have time in a day to even read.

As the number of AI articles and content approaches infinity, their value approaches zero.

[Fluff_Nutz]

@Christianz

For me I would never use AI at all. I even have two websites. My main and another that is much smaller and newer that I could technically use for AI as it only has roughly 70 articles on it. But, despite the success of AI sites right now. I cannot stoop to that level. AI should never be the answer. In fact I don't even like AI in search at all. It's technically classed as cheating. It serves no accomplishment. Just pure human weakness and proof that you cannot do it yourself without relying on technology that will never be 100% perfect.

[RedBar]

@ichthyous

I'm giving your robots.txt a trial on two well-trafficked sites to see what / if anything happens. I've been so well-scraped over the past 30 years that I doubt anything will jump out however curiosity has got the better of me.

Whilst overall traffic is down real-world business enquiries are still holding consistent plus, the important bit, new global clients' enquiries are still being generated.

To be honest I cannot imagine what it can be like for those working for someone else / under a performace contract and not having the complete inside information that I do, it must be extremely "exasperating".