SleepBot

[Bewenched]

Didnt ask for robots.txt
Hitting us from
35.188.190.161, 34.170.181.225,
and
136.115.236.76

google cloud

[Bewenched]

Oh and 34.59.220.193, 35.188.190.161

[lucy24]

:: shuffling through raw logs ::

Oh, there it is. Showed up quite suddenly near the beginning of this month:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; SleepBot/1.0; +http://sleepbot.com/) Chrome/131.0.0.0 Safari/537.36

mostly from 34.various and 35.various, once from 104.198.blahblah. Mostly https, a few http.

Oddly, mine have requested nothing but robots.txt, where they received the minimalist
Disallow: /
version, partly but not exclusively because of the Chrome in the UA. (I've got an environmental variable lying_bot for this purpose.)

Site linked in the UA--yes, really http--does not appear to have been updated since January 2008.

[not2easy]

34.59.220.193 = part of 34.4.5.0 - 34.63.255.255 which are directly allocated, often a sign of hosting IPs. A number of CIDRs are included:

34.4.5.0/24
34.4.6.0/23
34.4.8.0/21
34.4.16.0/20
34.4.32.0/19
34.4.64.0/18
34.4.128.0/17
34.5.0.0/16
34.6.0.0/15
34.8.0.0/13
34.16.0.0/12
34.32.0.0/11

34.170.181.225 (Direct Allocation)
34.128.0.0 - 34.191.255.255
CIDR: 34.128.0.0/10

136.115.236.76
136.112.0.0 - 136.127.255.255 (Direct Allocation)
CIDR: 136.112.0.0/12

Cloud servers (like 35.188.190.161)
35.184.0.0 - 35.191.255.255
CIDR: 35.184.0.0/13

[SumGuy]

When I get hits from googleusercontent (which these seem to be) it means that I'm not (yet) blocking that particular /16. But once is enough and so yes that /16 goes into my IP blocking list.

[shawnb61]

Why not block all of AS396982 - Google Cloud?

No (uh... extremely few...) end users there. Only "businesses", like this odd duck SleepBot, that seems to be systematically attempting to get all my images, via an awfully buggy script that cannot work... I'm not sure I see value in having rando businesses crawl my site like this.

I haven't seen googlebot work from AS396982 - though that's possible.

Just a thought.

Still an awful lot of random crawling going on. Last year, it was Brazil & Vietnam. Over the last few weeks, the questionable activity on our site is 95% Google Cloud (AS39698) & Microsoft (AS8075).