Annoying Crawler, Bot, or what?

Malicious traffic from a certain ISP/network

[PeterTroyWilliams]

Hi Folks, Il try this again, and I hope I am posting this in the right place.
My site is being hit by Oculus Networks repeatedly , which gives our site a high bounce rate. They seem to be in the 172.252.***.*** range and a few others. They come to the site, and hit the same page about 30 times, leave and come back again doing the same thing. They all are Linux, Chrome 149.0, 1920x1080. It is annoying, as we use some free services at our site that here limited by hits, before they are not free anymore. I do use cloudflare , and have attempted to block them through cloudflare which dont seem to be working. Any suggestions would be greatly appreciated!
~Pete

[lucy24]

have attempted to block them through cloudflare which dont seem to be working

This is where we will need details. *Exactly* what have you tried, and what if anything is the result? (I don't personally speak cloudflare, but plenty of others do, so be specific.)

some free services at our site that are limited by hits, before they are not free anymore

Does it go by total requests, or only by successful requests? In some cases, blocking a request doesn't really save your server much work, unless the pages themselves are quite hefty. I'm assuming your robots, like most, are only requesting pages, no supporting files--or at least, no images.

<tangent>
In recent months, I too have been much vexed by robots requesting the same page over and over--generally clusters of requests from all different IPs and UAs, rather than multiple requests from the same source. Short of getting stricter about oldish browsers, and blocking the rare IP that is a server farm/colo rather than human ISP, there's not a ### thing I can do.
</tangent>

[PeterTroyWilliams]

Not sure if they are. The all have the same thing in common, Oculus Networks, which is a proxy I think,and the browser specifics. I have gone into cloudlflare and blocked individual ip adresses, (ip.src eq 40.223.131.239 and ip.src eq 172.121.149.193 and ip.src eq 40.223.203.229 and ip.src eq 40.223.213.113 and ip.src eq 40.223.212.140 and ip.src eq 172.252.237.166 and ip.src eq 40.223.127.209 and ip.src eq 40.223.219.108) but they just keep bouncing around the ips. so maybe be question would be how to configure a complete range?

[lucy24]

how to configure a complete range?

Well, that would depend on Cloudflare. Do they recognize IPv4 blocks in the form 162.121.128.0/19, or wider ranges in the form 172.252 ? (i.e. the same format you would use in your htaccess / config / IIS equivalent.)

Anything involving proxies or ISP ranges means you also risk blocking legitimate humans, so there's a tradeoff. If your robots are anything like mine, there won't be any obvious header deficits. Do you have the option of blocking by User-Agent? This too is risky, since some humans will be stuck in older browsers.

[not2easy]

Just picking out one of those - 40.223.0.0 - 40.223.255.255 = CIDR: 40.223.0.0/16 which was an Eli Lily range but WHOIS shows it as reallocated to IXPO and used as a proxy of Oculus Networks.

172.121.149.0 - 172.121.149.255 = CIDR: 172.121.149.0/24 (also IXPO and used as a proxy of Oculus Networks. )
Same thing with 172.252.237.0 - 172.252.237.255 = CIDR: 172.252.237.0/24

As mentioned, it does not help much blocking a single IP as the proxy can use another IP in the same range. You're stuck blocking every user of the proxy range or playing constant whack-a-mole. I don't use Cloudflare so I'm of no help there, but there are many CF users around who might have actual useful suggestions.

[PeterTroyWilliams]

Yes, they recognize they recognize IPv4 blocks in the form 162.121.128.0/19 It can be configured to pose a captcha rather than a block, making me really wonder now if by the capta choice, if it is still actually connecting to the site, making a "hit". Again, my only issue is the bounce rate , and the annoyance of statistics being all trown out of wack.

[PeterTroyWilliams]

Thank you. Yes these are the IP ranges of the offender. I will see if I can use Cloudflare to block the complete range.

[PeterTroyWilliams]

Thanks. I went into cloudflare and made a security rule to block those CIDR numbers and a few more, I also blocked the associated ASN. Ill know tomorrow . I learned what all that means today! thanx again.

[PeterTroyWilliams]

So nice! Im seeing normal hits again, instead of Oculus hitting the same page with different IPs all day. I have seen this before, and when your sites bounce rate goes up, seems google ranking goes down. I hope this fixes the issue. Thanks all, and especially to not2easy!