Annoying Crawler, Bot, or what?

Malicious traffic from a certain ISP/network

[PeterTroyWilliams]

Hi Folks, Il try this again, and I hope I am posting this in the right place.
My site is being hit by Oculus Networks repeatedly , which gives our site a high bounce rate. They seem to be in the 172.252.***.*** range and a few others. They come to the site, and hit the same page about 30 times, leave and come back again doing the same thing. They all are Linux, Chrome 149.0, 1920x1080. It is annoying, as we use some free services at our site that here limited by hits, before they are not free anymore. I do use cloudflare , and have attempted to block them through cloudflare which dont seem to be working. Any suggestions would be greatly appreciated!
~Pete

[shawnb61]

Curious that so many robots claim to be Mac, when the Mac has traditionally been less prone to infection of any kind. Do they think that this alone lends plausibility?

I really think they just randomly choose among valid UAs. Millions to choose from... I often see UAs vary even from the same IP, like they're rotating thru a set.

I also think they keep track of what works - by site. After I blocked very old browsers, those requests dropped in frequency. Not all crawlers got the memo, I still get some, but, the numbers are down significantly after blocking.

I've seen that a number of times, another example was blocking PHPSESSID= URLs. After blocking, the volume of those requests dropped.

So any benefit I got along those lines was temporary...

[edited by: shawnb61 at 5:14 pm (utc) on Jul 22, 2026]

[shawnb61]

My site is being hit by Oculus Networks repeatedly
...
Any suggestions would be greatly appreciated!

If you can block by ASN, then block their entire ASN - AS398781.

If you cannot block by ASN, then block ALL their CIDRs. There's about ~400 total, including the 172.252s.

[lucy24]

I just checked, and, years later, that crazed old metalhead still has that odd useragent...

For genuinely niche cases, there's always

RewriteCond %{HTTP_USER_AGENT} any-absurdly-old-browser
RewriteCond %{REMOTE_ADDR} !some-specific-ip
RewriteRule . - [F]

Unless, of course, your crazed old metalhead also uses some antiquated connection method (satellite? dialup?) that changes IP on every request.

[SumGuy]

I've had another conversation with a bot about that Intel Mac OS thing. Leave it to an AI bot to only see half the story.

I searched my logs for "Intel Mac OS X 11" and found some. But strangley they were from Dec 2020 to May 2021.

I asked a bot about this, the answer was the change from OSX 10 to 11 in the user-agent was breaking things, and they reverted back to reporting 10_15_7 to fix that and also to add a privacy layer (anti-fingerprinting technique). So from May / June 2021 till today all you'll ever see (in the user-agent string) is Intel Mac OS X 10_15_7. This means you WILL see newer versions of Chrome paired with that OS X version.

I haven't done a log scan yet to see what just what versions of Chrome are turning up today paired with Intel Mac OS X 10_15_7 and hence to see if they are indeed up to 145 - 150 or not.

[shawnb61]

Yep, I'm seeing everything up thru Chrome/150 alongside "Intel Mac OS X 10_15_7".

The bots are trying to NOT look like bots... One of the easiest ways to do so is to use real UAs. There are literally millions to choose from.

[SumGuy]

There are many UA's to choose from, they they do have time-limited lifespan. As long as the Chrome version keeps changing and is reported in the UA, it's a useful way to ID the bots using residential proxies. Use the accept-language string when necessary.

For me, these bots always come for my pdf files, and I also use that as well as the presence or absence of a referrer.

[shawnb61]

I'm leaning towards querying whois info...

One of my bad guys is Datacamp. Lots of real users. Lots of crawlers. Would be nice to easily separate them.
- The good traffic from Datacamp, whois all points right back to Datacamp
- The bad/crawler traffic from Datacamp, whois almost always shows a NetType of "Reallocated" or "Reassigned", and the org info points to someone like Internet Utilities or IPXO, but some others as well. Leased out.

Gotta automate this & see how consistent it is over time...

[SumGuy]

I asked a bot -> Would a website owner / operator expect to see page or site requests FROM a Datacamp or CDN77 IP address?

Answer: Yes—a website operator can expect some requests to originate from DataCamp/CDN77 IP space, especially if their infrastructure (or caching/edge services they use) routes traffic through CDN77/Datacamp. However, it’s not guaranteed that all requests you see will be “real end users” coming directly from their home/office ISP. A CDN in particular can cause the source IP seen by your logs to be:

- a CDN edge / caching IP (common), or
- an IP associated with the CDN’s network/PoPs, depending on the setup.

So, in practice: if you observe traffic from IPs that belong to DataCamp Limited (CDN77), that’s consistent with legitimate CDN/edge behavior and/or other network services they provide.

(I got some clarification on the "they" in the answer above)

“They” is the website’s operator / their site’s infrastructure—i.e., the party that has the website set up to use DataCamp/CDN77 services (their CDN/edge configuration), so requests end up being served from/handled by DataCamp/CDN77 IPs. The browser/client is just making requests; the CDN/edge layer determines what source IP the website logs for those requests.

So Shawnb61, is it possible that your site hosting service/network is using Datacamp/CDN77 somewhere in it's network topology?

I've also seen garbage from datacamp, my hosting infrastructure does not involve them in any way, they don't fit my perception of an IP space where legit human browser activity can originate from, hence I have long since blocked all datacamp IP space.

-------------

Note: Earlier in my conversation with the bot, I asked -> Would residential or corporate customers use Datacamp or CDN77 as a proxy or vpn to browse the web? Is Datacamp (or CDN77) an ISP in terms of being a "last mile" internet connectivity provider?

Answer: Usually no: DataCamp (and CDN77 by branding/company relationship) is not a “last‑mile ISP” in the sense of being the entity that provides the final access line into your home/business (that’s typically a local access ISP/telecom using fiber/cable/DSL/fixed wireless).

But CDN77 can be involved in “last‑mile–adjacent” performance, because it operates CDN/network infrastructure (including “embedded PoPs” inside ISP networks) intended to make traffic reach users faster by serving content closer to them. That affects what people experience as “last mile,” even though it’s not the same as being the household’s access provider.

What they are for: CDN and edge delivery (caching content closer to users, DDoS protection, performance, secure delivery). Their “client” use is mainly about delivering your content/web traffic efficiently, not about tunneling a user’s web browsing through their network like a VPN/proxy.

[shawnb61]

I'm talking about AS60068 here... DataCamp, operating on AS60068, is their ISP service, not their CDN service. Datacamp does provide CDN, but I believe that is on one of their many other ASNs.

As noted above, for DataCamp AS60068, all the good, clearly end user, traffic goes straight thru DataCamp. All the heavy crawl traffic looks like it goes thru IPs they have leased out or reallocated somehow, according to whois. I've seen this on some other ISPs.

I thought I was on to something, but looking at more ISPs, this didn't hold up. All good and bad traffic for AT&T, for example, went straight thru AT&T and did not appear to be leased out or reallocated. No silver bullet.

I did get a subscription to Spamhaus PBL. Spot checking via that list, all of these IPs, for good & bad behavior thru ISPs, are listed as invalid email servers. My understanding is that the PBL sources this info from the ARIN "residential only" flag - because if residential, they shouldn't be sending email. I was hoping it would help differentiate between residential user IPs & datacenters out there. Again, no silver bullet...

[shawnb61]

I hadn't even run across CDNs as crawlers before...

But for the last two days, by far my biggest nuisance is Akamai. Kiddie scripters (looking for every old wp vulnerability), injection/hack attempts, crawls... All Akamai.

[shawnb61]

Back to the question of identifying residential proxies raised here...

I have found something that appears to be working, at least for our site. Our site is a forum, and the bots are targeting topics, messages & boards. The thing is, those are the ONLY requests MOST of them make... They're not reading a page, then testing all the links. The requests are extremely homogenous - they are literally cycling thru topics & boards & messages.

This is very different from registered users & real guests. People search, they look at profiles, they navigate around, and they load templates & other site resources.

So... I've been experimenting with tracking two buckets, per all inbound /24s: total requests, & crawl_targets. Crawl_targets count how many times that /24 requested a topic, board or message with a GET. Keep rolling totals over the last X minutes in a memory-resident DB.

If crawl_targets / total requests > 90%, for 100+ requests, that's a bot. In fact, the ONLY way a human can do that is to directly edit their browser URL 100 times... The crawl_target/request ratio is about 5% for actual people.

Benefits:
- It very effectively discerns between /24s used by real users & bots within ASNs for residential providers.
- It catches "slow crawlers". I see lots of those, /24s that only request a topic every 2-10 minutes, perpetually...
- ***It catches lots of hackers & sql injection attempts.*** They're doing the same thing as the slow crawlers - completely flying under the radar due to the low volume.

Limitations:
- It clearly won't catch all bots. I sometimes see bots that do stupid stuff like read robots.txt 100K times a day... Just the crawlers/scrapers.
- It won't handle the situation where residential proxies lease out IPs of household gateways, etc. In those instances, the users and bots would be in the same /24s. I have not seen evidence of this in my data; they supposedly exist out there though...

I have never seen a real user and a bot in the same /24... Yet...

Still prototyping & analyzing the traffic, not live yet.

[SumGuy]

I think you've said that you haven't been able to get a Spur account to test some IP's. Post a few IP's here or maybe PM me a list of a dozen or so and I can run them on Spur and tell you what they come back with.

BTW, on Aug 19 @ 12 noon ET Spur is doing a live webinar - Beyond the Great Firewall - Investigating China’s Global Proxy Infrastructure.

[thecoalman]

Our site is a forum...

Same here, it's a scourge. One of the most effective Cloudflare rules I have is:

Field: URI
Operator: Wildcard
Value: *&sid=*

AND

Field: Referer
Operator: Equals
Value: <blank>

AND

Field: Cookie value of
Operator: Does not exist
Value: yourcookiename_sid

Action: Managed Challenge

This is specific to phpBB. With the exception of identified bots(Google) the SID parameter always appears in on page links if there is no session or session has expired. 74K blocks in last 24 hours with 0.07% successfully passing the challenge.

[shawnb61]

Post a few IP's here or maybe PM me a list of a dozen or so and I can run them on Spur and tell you what they come back with.

Thanks.

Here are some examples. I'll focus on Cox Communications here, AS22773. Cox is a good example of a residential internet provider that also presents significant bot activity. I cannot block Cox at the ASN level, because I have a LOT of users who use Cox.

In an ideal world, we'd be able to tell Cox user IPs from Cox bot IPs. I'll provide examples of both, so we can see if there is something that differentiates them.

Cox Communications, user IPs:
72.196.120.48
70.175.48.90
68.230.127.135
72.200.76.24
68.224.138.139

Cox communications, bot IPs:
156.249.23.74, 156.249.23.102, 156.249.23.157 (entire /24)
156.248.31.233, 156.248.31.166, 156.248.31.5 (entire /24)
45.206.80.22, 45.206.80.214, 45.206.80.246 (entire /24)
154.198.17.122, 154.198.17.200, 154.198.17.14 (entire /24)

The user IPs are identified in whois as all directly allocated.

The bot IPs, on the other hand, all run thru Seychelles.

I don't think that's enough to assume they're leased out to bots, though... I see too many bot IPs that are directly allocated, and I often see end user activity with NetTypes of 'reallocated' or 'reassigned' or otherwise not clearly directly allocated. It just doesn't seem to be solid enough to identify bad IPs on its own, without looking at behavior as well.

[shawnb61]

Same here, it's a scourge.

Yep!

This is specific to phpBB. With the exception of identified bots(Google) the SID parameter always appears in on page links if there is no session or session has expired. 74K blocks in last 24 hours with 0.07% successfully passing the challenge.

We use SMF. A heads up - PHP, in 8.4, has deprecated the passing of the session ID via URL in this fashion. It will be removed completely in PHP 9.0. SMF has removed this, so, it is now safe to simply block all uses of PHPSESSID/sid, since SMF isn't doing it, it's just bots being bots.

I believe phpBB still supports this, so, you are challenging end users who have cookies restricted (since the session can't be passed by cookie, it's passed by url). Sometime before PHP 9.0, phpBB will also need to drop support for URL based session IDs. At that point you can just block 'em all outright.

A side note on session IDs via URL... It can double-up on session reads/writes, because your app tries to use what was passed, finds it doesn't exist, and has to recreate a new php session and start over - with every request... If you have DB-based sessions, that means that your guests are causing 2x the normal guest I/O... These bots that hit you with hundreds of thousands of &sid= URLs can spike CPU, & even bottleneck DB log activity.

When I've looked closely at these, the session IDs used by these bots were ALWAYS bogus... Random... As if they were trying to spike your CPU...

[thecoalman]

When you have "free" game developers and companies even offering to pay the consumer to use their residential IP as proxy I'm not sure how you go about blocking bots en-masse. This is really something the ISP's need to get a handle on.

The bot IPs, on the other hand, all run thru Seychelles.

These were a particular PITA with Cloudflare. If they get to last rule I have in Cloudflare I allow traffic from US and everyone else gets challenged. Since they were Cox IP's it was allowing them through as being from US. It's the only blocks I have in place for IP's..

[thecoalman]

I believe phpBB still supports this,

With phpBB on page links always have SID when there is no session or the session has expired for logged in user. It's just the first page for logged in user or guest accepting cookies. The only exception to that is identified bots. It's being removed in phpBB4 which is just around the corner, it's in beta 2 version right now.

You are correct it can cause performance issues with DB, there is few things being done to reduce issues like shorter session length for guests.

[shawnb61]

With phpBB on page links always have SID when there is no session or the session has expired for logged in user.

Yep. Just be aware, the PHP support for SID via URL was deprecated in PHP8.4 & is going away entirely in PHP9.0; it is now considered an unsafe practice. phpBB will need to issue a patch. What this will probably mean is that support for users who block/disable cookies will be limited going forward.

[shawnb61]

Post a few IP's here or maybe PM me a list of a dozen or so and I can run them on Spur and tell you what they come back with.

The above examples for Cox were a bit of a best-case scenario:
- User IPs as shown in whois were directly allocated
- Bot IPs as shown in whois were not...

Here are some examples that are the other way around...

Human user IPs (not directly allocated):
87.121.92.0 (Launtel)
115.186.198.0 (Superloop)
103.46.209.0 (Easyconnect)
31.129.155.0 (Home Computer Networks - RU)
146.75.136.0 (Fastly - legacy)
119.12.211.0 (IP Star)
106.51.78.0 (CABLELITE - IN)

Bot IPs, crawls & hack attempts (directly allocated):
131.222.252.0 (entire /24 - Highspeed - legacy)
72.14.199.0 (entire /24 - Google - same URL over & over....)
47.128.55.0 (entire /24 - Amazon - hack attempts)
54.252.183.0 (entire /24 - Amazon - hack attempts)
144.76.22.0 (entire /24 - Hetzner - legacy)
74.112.232.0 (entire /24 - Dauphin - hack attempts)

"legacy" = pre-RIR, I see both bots & humans in "legacy" IP ranges.

I have actual users who access via Amazon AS14618, or I'd cut the whole ASN off entirely.

In an ideal world, we could identify some rules/attributes that would tells us which IPs are leased to a residential proxy.

Whether the IP as been reallocated in some way is a clue, but not definitive, and using that info will lead to blocking real users. Certain parts of the world (Australia) it looks like all my users come in via some form of reallocated/reassigned IP blocks.

[thecoalman]

phpBB will need to issue a patch.

phpBB has it's own session management, regardless as I already mentioned sid in url is being removed in phpBB4 mostly because of the unidentified bots. The bot gets a bunch of links with sid, the next requests for those links comes from completely different IP's. Session validation fails and new sid is appended to links for those requests. Rinse and repeat.

[SumGuy]

None of your IP's came back from spur as being ID'd as a proxy, but there were some results I don't understand:

70.175.48.90 not-a-proxy (but client behavior = "GFW_GENERIC_PROXY_USER" ?)
72.200.76.24 not-a-proxy (but client behavior = "GFW_GENERIC_PROXY_USER" ?)

156.249.23.0/24
Spur says no for all those, scamalytics says they're assigned to "Bunny Technology LLC" (scamalytics also has a residential proxy "yes/no" result field but I don't know how robust it is compared to spur).

156.248.31.0/24 (entire /24)
45.206.80.0/24 (entire /24)
Spur says not a proxy, Scamalytics says Cox - also not a proxy.

154.198.17.0/24 (entire /24)
Spur says no for all those, scamalytics also says no, and says they're assigned to "Bunny Technology LLC"

154.198.0.0/16 is announced under AS35916 "MULTACOM CORPORATION" but 154.198.17.0/24 is announced under AS22773 Cox. I find it fishy when a /24 has it's own announcement.

I've found these:
AS6207 Bunny Communications
AS5065 Bunny Communications
AS399073 BUNNY TECHNOLOGY LLC
AS39600 BUNNY TECHNOLOGY LLC

But unless I missed it, none of them contain 154.198.17.0/24 or 156.249.23.0/24.

What I find is that bots or IP's performing site probing or trying wordpress hacks never come back as being ID'd as residential proxies. What were those IP's doing on your site? And what was their user-agent? Were they hitting an interior page or resource without a plausible (or any) referer?

[lucy24]

154.198.0.0/16

Unkind query: Has anyone ever had legitimate visitors from the Seychelles?* The amount of IP space seems wildly out of proportion to the islands’ total population of

:: google, google ::

135,000.

* Didn't bother to re-check, but I've got 154.192.0.0/11 (/eleven) labeled Seychelles.

[SumGuy]

Seychelles is part of Afrinic. According to Afrinic, they've been assigned 31,210 /24 IPv4's (not looking at IPv6 here). That's about 8 million IP's across 18 ASN's.

154.192.0.0/11 is registered to Cloud Innovation, showing a Seychelles contact address. From a website operator POV, Seychelles is garbage. It's the internet hosting equivalent to an offshore banking country.

But I would love to go on vacation there. Seychelles has about 7500 hotel rooms across the entire country, so you can figure that at any one time there could be 15k people that are viable visitors to your website from Seychelles. And those people are likely to be high net-worth.

[shawnb61]

scamalytics says they're assigned to "Bunny Technology LLC"

Scamalytics lists the ISP as Cox, but the org as Bunny.

Sure looks like a residential proxy to me. Or some other IP leasing relationship that amounts to the same thing between Bunny & Cox.

Funny, if you lookup their website, bunnytechnology dot net, it looks all professional & legit... Until you notice their testimonials are by "John Doe" and "Jane Smith"...

What I find is that bots or IP's performing site probing or trying wordpress hacks never come back as being ID'd as residential proxies.

Yep. The designation, as reported in all these tools, is meaningless. I suspect it's self-reported and voluntary, and frankly, why would they do that. It would defeat their business model. It's how they make money, so they obfuscate the relationship, specifically to make their bots harder to detect & block.

I believe Scamalytics attempts to detect some of these attributes by observing in & out traffic. That's why their descriptions are almost always "xxx appears to be yyyy based on IPs where we have visibility".

What were those IP's doing on your site?

Crawls & hack attempts. The lists below are not filtered other than by IP. This is what a string of consecutive GETs looks like... Anywhere from 2/hour to thousands/hour. They all point to real topics/boards/messages, straight outta my sitemap.

Almost all of these bots are doing the exact same thing... I could give you identical pages just like this from thousands of other IP ranges, completely different ISPs, on any given day.

This is exactly why my crawl_target ratio approach would work for us. The only way a user could reproduce this would be to edit their browser URL over & over.

Crawls:

156.249.23.74GET /index.php?topic=26200.25 HTTP/2.0
156.249.23.102GET /index.php?topic=18017.0 HTTP/2.0
156.249.23.157GET /index.php?topic=12909.0 HTTP/2.0
156.249.23.87GET /index.php?topic=27251.0 HTTP/2.0
156.249.23.157GET /index.php?topic=18598.0 HTTP/2.0
156.249.23.220GET /index.php?topic=20238.25 HTTP/2.0
156.249.23.6GET /index.php?topic=25203.75 HTTP/2.0
156.249.23.151GET /index.php?topic=34358.0 HTTP/2.0
156.249.23.108GET /index.php?topic=25588.0 HTTP/2.0
156.249.23.207GET /index.php?topic=28926.0 HTTP/2.0
156.249.23.155GET /index.php?topic=21549.0 HTTP/2.0
156.249.23.122GET /index.php?topic=21715.0 HTTP/2.0
156.249.23.90GET /index.php?topic=20910.75 HTTP/2.0
156.249.23.25GET /index.php?topic=15866.0 HTTP/2.0
156.249.23.230GET /index.php?topic=39594.0 HTTP/2.0
156.249.23.46GET /index.php?topic=4299.0 HTTP/2.0
156.249.23.119GET /index.php?topic=39827.0 HTTP/2.0
156.249.23.64GET /index.php?topic=18114.0 HTTP/2.0
156.249.23.60GET /index.php?topic=18017.0 HTTP/2.0
156.249.23.76GET /index.php?topic=18791.0 HTTP/2.0
156.249.23.144GET /index.php?topic=153.0 HTTP/2.0
156.249.23.124GET /index.php?topic=39124.0 HTTP/2.0
156.249.23.106GET /index.php?topic=26781.0 HTTP/2.0
156.249.23.181GET /index.php?topic=34796.0 HTTP/2.0

Hack attempts:

74.112.232.57GET /index.php?topic=%27%29%20AND%20%28%27bfslet%27%3D%27bfslet%27%27 HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%27%20ORDER%20BY%201--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%20AND%20EXTRACTVALUE%281699%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%281699%3D1699%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%27%20ORDER%20BY%201000--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%27%20AND%20EXTRACTVALUE%286827%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%286827%3D6827%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%20ORDER%20BY%201--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%25%27%20AND%20EXTRACTVALUE%285232%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%285232%3D5232%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%20ORDER%20BY%201000--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%22%20AND%20EXTRACTVALUE%282234%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%282234%3D2234%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%27%20UNION%20ALL%20SELECT%20%27yrzrhifkuxktuwtgydjvtzsluweowhuf%27--%205jd009 HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%25%22%20AND%20EXTRACTVALUE%288563%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%288563%3D8563%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=-36096.0%27%20UNION%20ALL%20SELECT%20%27yrzrhifkuxktuwtgydjvtzsluweowhuf%27--%205jd009 HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%60%20AND%20EXTRACTVALUE%282506%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%282506%3D2506%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%20UNION%20ALL%20SELECT%20%27yrzrhifkuxktuwtgydjvtzsluweowhuf%27--%205jd009 HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%29%20AND%20EXTRACTVALUE%285485%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%285485%3D5485%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=-36096.0%20UNION%20ALL%20SELECT%20%27yrzrhifkuxktuwtgydjvtzsluweowhuf%27--%205jd009 HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%27%29%20AND%20EXTRACTVALUE%289553%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%289553%3D9553%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%27%20UNION%20ALL%20SELECT%20%27yrzrhifkuxktuwtgydjvtzsluweowhuf%27%2CNULL--%206k8fl7 HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%25%27%29%20AND%20EXTRACTVALUE%286124%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%286124%3D6124%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=-36096.0%27%20UNION%20ALL%20SELECT%20%27yrzrhifkuxktuwtgydjvtzsluweowhuf%27%2CNULL--%206k8fl7 HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%22%29%20AND%20EXTRACTVALUE%283873%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%283873%3D3873%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%27%20UNION%20ALL%20SELECT%20NULL%2C%27yrzrhifkuxktuwtgydjvtzsluweowhuf%27--%20tpsscn HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%25%22%29%20AND%20EXTRACTVALUE%287793%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%287793%3D7793%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%60%29%29%29%20AND%20EXTRACTVALUE%284024%2CCONCAT%280x7e%2C%28%28SELECT%20%28ELT%284024%3D4024%2C1%29%29%29%29%2C0x7e%29%29--%20- HTTP/2.0
74.112.232.57GET /index.php?topic=36096.0%20UNION%20ALL%20SELECT%20NULL%2C%27yrzrhifkuxktuwtgydjvtzsluweowhuf%27%2CNULL--%2053qh7s HTTP/2.0

And what was their user-agent?

These bots all rotate among valid user agents. They're trying to look like people; they wouldn't give themselves away that easily.

74.112.232.57Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
74.112.232.57Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0
74.112.232.57Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15
74.112.232.57Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15
74.112.232.57Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0
74.112.232.57Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15
74.112.232.57Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0
74.112.232.57Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0
74.112.232.57Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0
74.112.232.57Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0
74.112.232.57Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0
74.112.232.57Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
74.112.232.57Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
74.112.232.57Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
74.112.232.57Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0
74.112.232.57Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0
74.112.232.57Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15
74.112.232.57Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
74.112.232.57Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
74.112.232.57Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
74.112.232.57Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0
74.112.232.57Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0

Were they hitting an interior page or resource without a plausible (or any) referer?

These are all public pages. It's our site. They're only hitting exposed pages.

Referrer is usually listed as OUR site or blank. Lotsa blanks. Can't block blanks, though, as initial site load has a blank referer for real folks. Also some privacy solutions blank that out. I see LOTS of valid user activity with a blank referer.

All this interest in my humble little guitar effects site. 200K-300K hits a day, 95%+ bots. I'm convinced it's all for AI training. It's where the money is.

[shawnb61]

Actually, a correction to my note above: residential proxies are not required to report anything, to anyone, anywhere...

So... The 'residential proxy' designation is a best-guess by these services, scamalytics or spur, etc., based on observing traffic.

And, it appears, they are bad at it.

[thecoalman]

Just noticed something that is absolutely crazy. I'm seeing hundreds of requests per hour for:

posting.php?mode=quote&p=12345

This is from quote button on a post if it's not obvious. It's available to guests but they get login/registration page if they aren't logged in.

12345 is just random number but not the same one each request, thousands of IP's with just a few requests per IP spread out. multiple residential US networks and multiple UA's.

<sigh>

[lucy24]

This is from quote button on a post if it's not obvious. It's available to guests but they get login/registration page if they aren't logged in.

Does it come complete with plausible referer?

[thecoalman]

No referer and no cookie. I'm now Issuing a challenge with cloudflare when all three criteria met.

What is insane is the amount of IP's spread across so many networks for something they can't access.... and it's only thing they are trying to access.

It's all residential US IPs/networks but these generally get no challenge from me.

[SumGuy]

Scamalytics has ID's some or most of your hits as coming from "Bunny Technology LLC" even if they are Cox IP's. Because Cox has sub-assigned them to a third party, they don't qualify as being residential proxies - they would never be recruited into any of the various residential proxy networks. When you see a rash of hits from different IP's all coming from the same /24 (like 156.249.23.0/24) again you'd never see that from an authentic residential set of IP's.

go to bgp.he.net and do a search for "bunny" (quotes not needed). They don't all show up (like 156.249.23.0/24 doesn't show up) but it's a start.

[blend27]

re: from "Bunny Technology LLC" if so bunnytechnology.net, then....

from IPINFO data

ASNs: AS39600 and AS399073
Located in 50 Countries.

141.11.152.0/23 - HK - AS39600
91.124.194.0/23 - HK - AS39600
91.124.222.0/23 - HK - AS39600
45.195.156.0/24 - PH - AS39600
45.195.159.0/24 - PH - AS39600
66.80.2.0/24 - SG - AS39600
66.253.3.0/24 - SG - AS39600
64.145.2.0/24 - SG - AS39600
66.253.37.0/24 - SG - AS39600
66.93.65.0/24 - SG - AS39600
66.253.40.0/24 - SG - AS39600
64.145.14.0/24 - SG - AS39600
67.102.176.0/24 - SG - AS39600
155.229.16.0/24 - SG - AS39600
67.102.178.0/24 - SG - AS39600
155.229.87.0/24 - SG - AS39600
67.102.182.0/24 - SG - AS39600
155.229.208.0/24 - SG - AS39600
207.145.42.0/24 - SG - AS39600
216.254.93.0/24 - SG - AS39600
102.68.48.0/22 - SZ - AS39600
50.114.119.0/24 - US - AS39600
74.114.117.0/24 - US - AS39600
173.211.105.0/24 - US - AS39600
184.174.82.0/24 - US - AS39600
31.56.223.128/29 - AE - AS399073
31.56.223.136/31 - AE - AS399073
31.56.223.138 - AE - AS399073
31.56.223.140/30 - AE - AS399073
31.56.223.144/28 - AE - AS399073
31.56.223.160/27 - AE - AS399073
178.93.66.0/24 - AR - AS399073
178.95.111.0/24 - AR - AS399073
178.95.115.0/24 - AT - AS399073
178.95.120.0/24 - AU - AS399073
178.93.68.0/24 - AU - AS399073
178.93.69.0/24 - AZ - AS399073
178.95.121.0/24 - AZ - AS399073
178.93.70.0/24 - BD - AS399073
178.93.71.0/24 - BE - AS399073
178.93.72.0/24 - BG - AS399073
178.93.73.0/24 - BH - AS399073
178.93.75.0/24 - BO - AS399073
178.93.84.0/24 - BR - AS399073
178.93.85.0/24 - BY - AS399073
178.93.89.0/24 - CA - AS399073
178.93.92.0/24 - CH - AS399073
178.92.1.0/24 - CH - AS399073
178.93.94.0/24 - CL - AS399073
82.152.51.193 - CN - AS399073
178.92.9.0/24 - CO - AS399073
178.92.10.0/24 - CR - AS399073
178.92.3.0/24 - CR - AS399073
178.92.4.0/24 - CY - AS399073
178.92.25.0/24 - CY - AS399073
178.92.26.0/24 - CZ - AS399073
178.92.12.0/24 - CZ - AS399073
178.92.23.0/24 - DE - AS399073
178.93.114.0/24 - DK - AS399073
178.93.136.0/24 - DK - AS399073
178.93.135.0/24 - DO - AS399073
178.93.134.0/24 - DZ - AS399073
178.93.112.0/24 - DZ - AS399073
178.93.133.0/24 - EC - AS399073
178.92.55.0/24 - EC - AS399073
151.241.102.0/24 - EE - AS399073
178.92.51.0/24 - EE - AS399073
178.93.132.0/24 - EE - AS399073
178.93.99.0/24 - EG - AS399073
178.93.131.0/24 - EG - AS399073
178.93.130.0/24 - ES - AS399073
178.93.96.0/24 - ES - AS399073
178.94.188.0/24 - FI - AS399073
178.94.186.0/24 - FR - AS399073
77.107.105.0/24 - GB - AS399073
178.94.185.0/24 - GB - AS399073
82.152.51.192 - GB - AS399073
82.152.51.194/31 - GB - AS399073
82.152.51.196/30 - GB - AS399073
178.93.55.0/24 - GR - AS399073
178.94.181.0/24 - GR - AS399073
178.93.53.0/24 - GT - AS399073
163.5.220.0/24 - HK - AS399073
185.246.115.0/24 - HK - AS399073
81.168.126.0/24 - HK - AS399073
178.93.52.0/24 - HK - AS399073
185.36.204.0/24 - HK - AS399073
196.251.251.0/24 - HK - AS399073
89.213.139.0/24 - HK - AS399073
178.92.53.0/24 - HK - AS399073
193.142.22.0/24 - HK - AS399073
45.90.19.0/24 - HK - AS399073
89.213.152.0/24 - HK - AS399073
178.93.33.0/24 - HK - AS399073
62.72.185.96/27 - HK - AS399073
172.121.190.0/24 - HK - AS399073
217.145.78.0/24 - HK - AS399073
154.16.228.0/24 - HK - AS399073
62.72.185.128/25 - HK - AS399073
141.11.173.0/24 - HK - AS399073
46.236.242.0/24 - HK - AS399073
95.135.189.0/24 - HK - AS399073
102.217.104.0/24 - HK - AS399073
82.152.51.200/29 - HK - AS399073
145.79.165.0/24 - HK - AS399073
82.152.51.208/28 - HK - AS399073
154.127.60.0/24 - HK - AS399073
82.152.51.224/27 - HK - AS399073
82.153.159.0/24 - HK - AS399073
82.153.215.0/24 - HK - AS399073
151.243.214.0/24 - HK - AS399073
31.56.223.192/26 - HK - AS399073
31.6.51.0/24 - HK - AS399073
31.56.235.0/27 - HK - AS399073
31.56.223.139 - HK - AS399073
31.56.210.0/24 - HK - AS399073
31.56.212.0/24 - HK - AS399073
82.152.51.0/25 - HK - AS399073
82.152.51.128/26 - HK - AS399073
31.56.235.144/29 - HK - AS399073
31.56.223.0/25 - HK - AS399073
45.11.175.0/24 - HK - AS399073
31.56.235.44/31 - HK - AS399073
31.56.235.48/28 - HK - AS399073
31.56.235.64/26 - HK - AS399073
31.56.235.176/28 - HK - AS399073
31.56.235.192/26 - HK - AS399073
204.69.217.0/24 - HK - AS399073
109.176.193.0/24 - HR - AS399073
87.229.9.0/24 - HR - AS399073
178.93.47.0/24 - HR - AS399073
178.93.46.0/24 - HU - AS399073
178.93.45.0/24 - ID - AS399073
31.56.235.128/28 - ID - AS399073
31.56.235.46/31 - ID - AS399073
31.56.235.152/29 - ID - AS399073
31.56.235.160/28 - ID - AS399073
31.56.235.32/29 - ID - AS399073
31.56.235.40/30 - ID - AS399073
178.93.39.0/24 - IE - AS399073
62.72.185.0/26 - JO - AS399073
62.72.185.64/27 - JO - AS399073
207.145.12.0/24 - SG - AS399073
216.254.1.0/24 - SG - AS399073
64.145.1.0/24 - SG - AS399073
66.93.68.0/24 - SG - AS399073
64.145.10.0/24 - SG - AS399073
66.93.78.0/24 - SG - AS399073
66.93.248.0/24 - SG - AS399073
178.94.152.0/24 - SK - AS399073
178.94.153.0/24 - TH - AS399073
178.94.155.0/24 - TR - AS399073
178.94.156.0/24 - TW - AS399073
69.17.56.0/22 - US - AS399073
70.39.185.0/24 - US - AS399073
66.93.4.0/24 - US - AS399073
178.94.158.0/23 - US - AS399073
151.241.236.0/22 - US - AS399073
70.39.186.0/23 - US - AS399073
66.93.10.0/24 - US - AS399073
178.95.110.0/24 - US - AS399073
151.241.248.0/22 - US - AS399073
178.93.113.0/24 - US - AS399073
66.93.13.0/24 - US - AS399073
178.93.129.0/24 - US - AS399073
150.241.229.0/24 - US - AS399073
178.92.27.0/24 - US - AS399073
66.93.19.0/24 - US - AS399073
178.92.8.0/24 - US - AS399073
150.241.232.0/24 - US - AS399073
178.93.65.0/24 - US - AS399073
66.93.63.0/24 - US - AS399073
178.93.67.0/24 - US - AS399073
151.240.123.0/24 - US - AS399073
140.150.235.0/24 - US - AS399073
155.117.14.0/24 - US - AS399073
140.150.237.0/24 - US - AS399073
151.240.228.0/24 - US - AS399073
140.233.188.0/23 - US - AS399073
155.117.24.0/21 - US - AS399073
143.14.24.0/21 - US - AS399073
151.240.230.0/23 - US - AS399073
143.14.96.0/21 - US - AS399073
155.117.55.0/24 - US - AS399073
143.14.112.0/23 - US - AS399073
151.240.232.0/24 - US - AS399073
143.14.115.0/24 - US - AS399073
155.117.56.0/22 - US - AS399073
143.14.116.0/22 - US - AS399073
151.240.239.0/24 - US - AS399073
178.92.2.0/24 - US - AS399073
155.117.64.0/21 - US - AS399073
31.56.115.0/24 - US - AS399073
151.240.241.0/24 - US - AS399073
31.56.224.0/24 - US - AS399073
155.117.72.0/22 - US - AS399073
31.56.216.0/24 - US - AS399073
151.240.255.0/24 - US - AS399073
31.56.221.0/24 - US - AS399073
155.117.78.0/24 - US - AS399073
31.57.40.0/24 - US - AS399073
151.241.41.0/24 - US - AS399073
31.57.52.0/24 - US - AS399073
155.117.80.0/24 - US - AS399073
31.57.67.0/24 - US - AS399073
151.241.76.0/24 - US - AS399073
31.57.95.0/24 - US - AS399073
155.117.88.0/21 - US - AS399073
95.134.62.0/24 - US - AS399073
151.241.91.0/24 - US - AS399073
95.134.65.0/24 - US - AS399073
155.117.103.0/24 - US - AS399073
95.134.66.0/23 - US - AS399073
151.241.96.0/23 - US - AS399073
95.134.69.0/24 - US - AS399073
155.117.128.0/21 - US - AS399073
95.134.71.0/24 - US - AS399073
151.241.98.0/24 - US - AS399073
95.134.89.0/24 - US - AS399073
155.117.211.0/24 - US - AS399073
95.134.92.0/24 - US - AS399073
151.241.101.0/24 - US - AS399073
151.243.220.0/24 - US - AS399073
155.117.226.0/24 - US - AS399073
151.243.237.0/24 - US - AS399073
151.243.238.0/24 - US - AS399073
151.243.243.0/24 - US - AS399073
151.243.246.0/24 - US - AS399073
151.243.249.0/24 - US - AS399073
151.243.252.0/24 - US - AS399073
151.244.65.0/24 - US - AS399073
151.244.68.0/24 - US - AS399073
151.244.70.0/24 - US - AS399073
151.244.73.0/24 - US - AS399073
151.244.86.0/24 - US - AS399073
151.244.88.0/24 - US - AS399073
151.244.95.0/24 - US - AS399073
151.244.103.0/24 - US - AS399073
151.245.231.0/24 - US - AS399073
151.245.232.0/22 - US - AS399073
151.245.236.0/23 - US - AS399073
151.245.248.0/22 - US - AS399073
151.245.252.0/23 - US - AS399073
151.246.205.0/24 - US - AS399073
151.246.206.0/23 - US - AS399073
151.246.208.0/23 - US - AS399073
151.246.211.0/24 - US - AS399073
151.246.212.0/23 - US - AS399073
151.246.219.0/24 - US - AS399073
151.246.220.0/23 - US - AS399073
151.247.64.0/21 - US - AS399073
151.247.72.0/23 - US - AS399073
178.95.105.0/24 - UY - AS399073
178.95.106.0/24 - VN - AS399073
178.95.108.0/24 - ZA - AS399073