October 2023 Google Search Observations

[Cyril TechWebsites]

---

System: The following 3 messages were cut out of thread at: https://www.webmasterworld.com/google/5093320.htm [webmasterworld.com] by not2easy - 11:10 am on Oct 1, 2023 (atl -4)

---

Seeing further drastical drop on Saturday. I stopped producing new content and updating the old content. There is no sense anymore for me, feeling awful to say that, but my website was fully destroyed with the "helpful" update.

[Micha]

@cfiila Unfortunately, I also see that with the Ki content, often also unchecked and complete nonsense. But Google likes it.

[christianz]

I blocked Google Extended and all the other AI scrape-bots in my robots.txt about two weeks ago and I am wondering if this is having an effect. Has anyone else seen a decline in traffic after doing that?

Are you joking? How exactly? These bots are not giving you any visitors just the opposite.

Just as an anecdote - the site I did add these blocks to, shot up to all time high traffic shorty after doing it. Of course - it is in no way related to that. Just saying to ease your paranoia.

[ichthyous]

@christianz Do you think it's a stretch that Google would NOT algorithmically demote sites that deny it from using content to train Bard. Considering all the other tactics Google uses, it would not be a surprise for Google to want to 'motivate' us all into allowing it to use our content

[mysitegothithard]

My website was destroyed with this update. I lost 80% of my traffic and about 80% of my income. My website is a reference in my niche, and now I have been overtaken by several blogs that copy my content. I'm devastated. I'm from the Brazilian market and I'm anxiously waiting for some miracle to happen.

[christianz]

@ichthyous It would be a stretch, although NOT impossible. However, it would be too easy to spot this correlation. Robots.txt is public and nearly half of the internet is already doing this blocking. If that one half would magically get demoted, analysts/tools would pick it up.

If anything, I actually think this block is more likely to be POSITIVE ranking signal. It means you think you have something valuable content wise. And Google may scrape it (keep a copy from web crawler) in a special "valuable" folder for secret AI training etc.

[aabid099]

@mysitegothithard

If something happens because of our fault then it will be bearable but watching other crap sites having the same or ai content gets ranking, this is something not fair or bearable, and google community support executive never takes any responsibility they always try to find fault on your side.

[mosxu]

After $166 billion loss I predict a lot of updates

[mosxu]

80% plus zombies, someone has to end this party

[RedBar]

I blocked Google Extended and all the other AI scrape-bots in my robots.txt about two weeks ago and I am wondering if this is having an effect. Has anyone else seen a decline in traffic after doing that?

Well, yes I have, I did this about 4-6 weeks ago and have considered this too, is it purely coincidental or my supicious conspiracy theory running riot?

I am still ranking extrremely well for many keyword phrases and my inkling is still localisation is the main driver of this however does anyone really know even G?

[mosxu]

@RedBar

The more you block the better!

My honest opinion!

[christianz]

This robots.txt block is only beginning for me. In future I will be blocking IP addresses based on how many requests they do, to make scraping more difficult. May make exception for Google's/Bing's published crawler IPs.

But block in robots.txt is basic zero-downside and zero-difficulty thing that everyone should do, without exception.

Internet is locking down. More paywalls, content only after login etc. Look at X - they don't show any recent tweets unless I am logged in. Reddit talks about blocking Google etc. It will only get bigger.

The open web is turning into scraper/spinner hellscape. Soulless fake content written by AI for AI.

[cfiila]

Funny. On the help forum they always say that the fault is for not having clear information about the author, publisher, etc. That's usually what they claim.

However, the website being displayed has a generic author (Editor/Author/Site Name) with a reference who loves the subject.

It has no policy, no author data, nothing, nothing.

Google's joke!

[ichthyous]

The open web is turning into scraper/spinner hellscape. Soulless fake content written by AI for AI.

I see this as a potential pitfall of AI. The whole premise is based on unrestricted scraping of other people's data...but what happens when that data is withdrawn by more and more sites? A smaller and smaller dataset of data will be used to train the algo and it will produce 'anomalies'

[Dooku]

A smaller and smaller dataset of data will be used to train the algo and it will produce 'anomalies'

Just like the Ublock Origin browser add-on someone should start building a list of all the "AI Bots" out there with IP's.
Webmasters should then add a few pages of garbage and nonsense info to their websites.
Once any such AI bot comes knocking at your website it will be relayed and get fed that nonsense :-)
While normal users will see the actual pages......the bot's will be trained on sh|t :-)

[christianz]

@Dooku, yes this is great idea. It is a form of shadow-banning. The bot won't know it is banned so it won't hide (use a different bunch of IPs) and continue to absorb slightly messed-up data.

In that scenario you have to be careful not to treat Google that way or else not only will your Google index become nonsensical but also it will be a form of cloaking.

[mosxu]

[bleepingcomputer.com...]

Blocking IPs to become obsolete and that in the name of “user privacy”

[haramamba]

[bleepingcomputer.com...]
Blocking IPs to become obsolete and that in the name of “user privacy”

In upcoming phases, Google plans to adopt a 2-hop proxy system to increase privacy further.

Just block the proxies. Nobody likes proxied/VPN'ed visitors.

[haramamba]

In that scenario you have to be careful not to treat Google that way

Look for the googlebot ip range json, download it periodically and whitelist the ranges.

[Dooku]

Blocking IPs to become obsolete and that in the name of “user privacy”

The usual google BS. As google is controlling most of this infrastructure and seeing google's history in abusing user privacy in favor of it's user behavorial SE, I guarantee you they will collect and (ab)use this data. Nothing is for "free" at google.

This has nothing to do with user privacy. This is a first door(proxy) from which google can learn, control and then know what a user will be doing without the "end location" websites knowing where there traffic came from and thereby those websites loosing valuable commercial data. This is just diabolical......

I am already blocking VPN users through Cloudflare, it's also very easy to block those VPN networks using Nginx if you do not use Cloudflare.

Also as chritianz and haramamba noted, obviously the normal google bots should NOT be blocked. Therefore we do not need the normal google bot IP ranges. We only need the IP ranges of the "AI bots" and their user agents.

[haramamba]

We only need the IP ranges of the "AI bots" and their user agents.

Amazon AWS, MS Azure, G's cloud, OVH, Hetzner, Leaseweb. From 500 to 1000 ASNs of hosting companies. All of them host bots.
But now I see multiple broadband visitors who pass a simple captcha, load only one page and stop browsing. They look like some AI bots but you cannot stop them just by blocking ip ranges.

[Conro]

Found in the comment of Seroundtable. Next time someone tells you that your website is penalized because it's poorly done, show them this. Maybe they'll wake up

[imgur.io...]

[haramamba]

@Conro
Why the EAT is s**t? Because when I google for the "best passive speakers" I see The NY Times on the first place in serps.

[RubicCubed]

Google is getting ready to test a new "IP Protection" feature for the Chrome browser that enhances users' privacy by masking their IP addresses using proxy servers.

In other words, Google wants to mask IP addresses so advertisers can't identify the fake clicks Google is billing them for. I'm reaching the end of the rope here and can see where I soon just block everything related to Google. What's to lose when Google doesn't send any converting traffic? Not much really...

@Conro

The excessive ads shown in that screen shot is really nothing new, but may be new to that region. In the USA we've been dealing with this since the beginning of the year.

[ichthyous]

Google is getting ready to test a new "IP Protection" feature for the Chrome browser that enhances users' privacy by masking their IP addresses using proxy servers.

Competing with Safari, which already has been doing this for a long time. Searches come in as 'Icloud Private Relay' identified as USA/English and that's it. It's popular with users. This will make it difficult for us to really analyze the source of our traffic but nothing we can do about it if the tech giants decide to do this.

But I also see the point of those here saying it will stop them from being able to catch fake clicks coming from Google ads. I honestly don't know why anyone would pay two cents to Google for their crap traffic from ads anymore. My experience over and over has been that it's really worthless...it takes over 100 clicks to even get an inquiry submitted on my site let alone a sale. If it actually provided any results I would have been using it all along, but you do nothing but fight with Google about all the fake clicks they are charging you for. No thanks, anything else is better.

[Conro]

@haramamba strange Google no penalize the affiliate content of New York Times :D usually google penalize the affiliate content after few months with the excuse of spam, content not with the theme of site, eeat etc etc

[RedBar]

@Dooku

I am already blocking VPN users through Cloudflare

That's your perogative however I, like many others, use a VPN for perfectly valid reasons.

Supposedly 1 in 3 global users use one and in the USA 35%, potentially that's a lot of users you are blocking:

[cloudwards.net...]

[haramamba]

Supposedly 1 in 3 global users use one and in the USA 35%, potentially that's a lot of users you are blocking:

I suppose we are talking about blocking cheap or even free vpns/proxies that people typically use to hide their a$$es (hosted on Amazon, OVH, etc).
If a remote employee uses vpn to connect to a corporate network and then to browse your website, you will not even know that he uses vpn.
Yes, you may see some entries in the server logs like xxx.vpn.some-company.com but will you block traffic from Bosch/Ford/Microsoft corporate nets?

[EditorialGuy]

@haramamba strange Google no penalize the affiliate content of New York Times :D usually google penalize the affiliate content after few months with the excuse of spam, content not with the theme of site, eeat etc etc

Google may not be enthusiastic about affiliate content, but (at least in the past) it has said that affiliate links in informational content are okay.

To be fair to The New York Times, its Wirecutter reviews are legitimate editorial content. If you were to remove the affiliate links, the reviews would still be useful to a typical NYT reader.

Also, NYT Wirecutter reviews typically include multiple merchant links for each product (e.g., Amazon, Target, and Walmart, as opposed to just one vendor). I seem to recall a Google statement or spokesperson saying something positive about that multi-vendor approach in the last few months.

[Dooku]

Supposedly 1 in 3 global users use one and in the USA 35%, potentially that's a lot of users you are blocking

Sorry, have to disagree with you on this one and I have considerable data on all of my websites, including the ones I do NOT block VPN networks for comparison to see what exactly is going on.
You do know what Cloudwards is right, and that these guys are ONLY commercially motivated?! "See how many people using a VPN!, you should too!" Their statistics are rubbish and should be taken with a (BIG)pinch of salt. Especially the 35% in the USA is nonsense, half the (online)population don't even know what a VPN is and the remaining 90% can't be bothered.

Many of the VPN networks are nothing but hideaways for low life trying to do things on your website that no normal user does.
I am NOT blocking 100% of all VPN networks but I have my list of networks that contain nothing but useless traffic in the form of bots, (wannabee)hackers and fraudulent actors. Many orders placed using a VPN get flagged by my payment processors, not because of the customer using a VPN, but because of "inconsistencies" in their card/payment info.......and in about 80% they are correct(after doing my own checking also).

[RedBar]

@Dooku
I'm not tryiing to convince you of anything just simply presenting some data available. Whether it's accurate or not I have no idea but I do know personally of many, many people using VPNs who a few years wouldn't have had a clue what it is and many of those have been insisted upon by their employers, however I wouldn't be as quick as yourself to assume that so many people have such little knowledge about it and many other "things".