Annoying Crawler, Bot, or what?

Malicious traffic from a certain ISP/network

[PeterTroyWilliams]

Hi Folks, Il try this again, and I hope I am posting this in the right place.
My site is being hit by Oculus Networks repeatedly , which gives our site a high bounce rate. They seem to be in the 172.252.***.*** range and a few others. They come to the site, and hit the same page about 30 times, leave and come back again doing the same thing. They all are Linux, Chrome 149.0, 1920x1080. It is annoying, as we use some free services at our site that here limited by hits, before they are not free anymore. I do use cloudflare , and have attempted to block them through cloudflare which dont seem to be working. Any suggestions would be greatly appreciated!
~Pete

[shawnb61]

Curious that so many robots claim to be Mac, when the Mac has traditionally been less prone to infection of any kind. Do they think that this alone lends plausibility?

I really think they just randomly choose among valid UAs. Millions to choose from... I often see UAs vary even from the same IP, like they're rotating thru a set.

I also think they keep track of what works - by site. After I blocked very old browsers, those requests dropped in frequency. Not all crawlers got the memo, I still get some, but, the numbers are down significantly after blocking.

I've seen that a number of times, another example was blocking PHPSESSID= URLs. After blocking, the volume of those requests dropped.

So any benefit I got along those lines was temporary...

[edited by: shawnb61 at 5:14 pm (utc) on Jul 22, 2026]

[shawnb61]

My site is being hit by Oculus Networks repeatedly
...
Any suggestions would be greatly appreciated!

If you can block by ASN, then block their entire ASN - AS398781.

If you cannot block by ASN, then block ALL their CIDRs. There's about ~400 total, including the 172.252s.

[lucy24]

I just checked, and, years later, that crazed old metalhead still has that odd useragent...

For genuinely niche cases, there's always

RewriteCond %{HTTP_USER_AGENT} any-absurdly-old-browser
RewriteCond %{REMOTE_ADDR} !some-specific-ip
RewriteRule . - [F]

Unless, of course, your crazed old metalhead also uses some antiquated connection method (satellite? dialup?) that changes IP on every request.

[SumGuy]

I've had another conversation with a bot about that Intel Mac OS thing. Leave it to an AI bot to only see half the story.

I searched my logs for "Intel Mac OS X 11" and found some. But strangley they were from Dec 2020 to May 2021.

I asked a bot about this, the answer was the change from OSX 10 to 11 in the user-agent was breaking things, and they reverted back to reporting 10_15_7 to fix that and also to add a privacy layer (anti-fingerprinting technique). So from May / June 2021 till today all you'll ever see (in the user-agent string) is Intel Mac OS X 10_15_7. This means you WILL see newer versions of Chrome paired with that OS X version.

I haven't done a log scan yet to see what just what versions of Chrome are turning up today paired with Intel Mac OS X 10_15_7 and hence to see if they are indeed up to 145 - 150 or not.

[shawnb61]

Yep, I'm seeing everything up thru Chrome/150 alongside "Intel Mac OS X 10_15_7".

The bots are trying to NOT look like bots... One of the easiest ways to do so is to use real UAs. There are literally millions to choose from.

[SumGuy]

There are many UA's to choose from, they they do have time-limited lifespan. As long as the Chrome version keeps changing and is reported in the UA, it's a useful way to ID the bots using residential proxies. Use the accept-language string when necessary.

For me, these bots always come for my pdf files, and I also use that as well as the presence or absence of a referrer.

[shawnb61]

I'm leaning towards querying whois info...

One of my bad guys is Datacamp. Lots of real users. Lots of crawlers. Would be nice to easily separate them.
- The good traffic from Datacamp, whois all points right back to Datacamp
- The bad/crawler traffic from Datacamp, whois almost always shows a NetType of "Reallocated" or "Reassigned", and the org info points to someone like Internet Utilities or IPXO, but some others as well. Leased out.

Gotta automate this & see how consistent it is over time...

[SumGuy]

I asked a bot -> Would a website owner / operator expect to see page or site requests FROM a Datacamp or CDN77 IP address?

Answer: Yes—a website operator can expect some requests to originate from DataCamp/CDN77 IP space, especially if their infrastructure (or caching/edge services they use) routes traffic through CDN77/Datacamp. However, it’s not guaranteed that all requests you see will be “real end users” coming directly from their home/office ISP. A CDN in particular can cause the source IP seen by your logs to be:

- a CDN edge / caching IP (common), or
- an IP associated with the CDN’s network/PoPs, depending on the setup.

So, in practice: if you observe traffic from IPs that belong to DataCamp Limited (CDN77), that’s consistent with legitimate CDN/edge behavior and/or other network services they provide.

(I got some clarification on the "they" in the answer above)

“They” is the website’s operator / their site’s infrastructure—i.e., the party that has the website set up to use DataCamp/CDN77 services (their CDN/edge configuration), so requests end up being served from/handled by DataCamp/CDN77 IPs. The browser/client is just making requests; the CDN/edge layer determines what source IP the website logs for those requests.

So Shawnb61, is it possible that your site hosting service/network is using Datacamp/CDN77 somewhere in it's network topology?

I've also seen garbage from datacamp, my hosting infrastructure does not involve them in any way, they don't fit my perception of an IP space where legit human browser activity can originate from, hence I have long since blocked all datacamp IP space.

-------------

Note: Earlier in my conversation with the bot, I asked -> Would residential or corporate customers use Datacamp or CDN77 as a proxy or vpn to browse the web? Is Datacamp (or CDN77) an ISP in terms of being a "last mile" internet connectivity provider?

Answer: Usually no: DataCamp (and CDN77 by branding/company relationship) is not a “last‑mile ISP” in the sense of being the entity that provides the final access line into your home/business (that’s typically a local access ISP/telecom using fiber/cable/DSL/fixed wireless).

But CDN77 can be involved in “last‑mile–adjacent” performance, because it operates CDN/network infrastructure (including “embedded PoPs” inside ISP networks) intended to make traffic reach users faster by serving content closer to them. That affects what people experience as “last mile,” even though it’s not the same as being the household’s access provider.

What they are for: CDN and edge delivery (caching content closer to users, DDoS protection, performance, secure delivery). Their “client” use is mainly about delivering your content/web traffic efficiently, not about tunneling a user’s web browsing through their network like a VPN/proxy.