OVH Hosted Bots & Scrapers

[JamieW]

Over the past 24 months, the web server logs across a dozen websites I manage have a high percentage of traffic coming from the OVH data center. This traffic is coming in via numerous IP addresses assigned to OVH. Since the volume of traffic is dramatically larger than the traffic coming from legitimate ISPs (ATT, Verizon, Charter, Comcast, Shaw, etc), I have the impression that the traffic from OVH is due to bots/scrapers hosted at the OVH data center cloud servers. I have attached a log file of IP addresses hitting the servers from OVH.

Does anyone else have an opinion on incoming web server traffic from OVH? Does anyone else block all the traffic from these IP addresses?

129.213.67.0/24
142.44.149.0/24
142.44.167.0/24
144.217.64.0/24
144.217.66.0/24
149.56.241.0/24
151.80.19.0/24
154.21.212.0/24
158.69.124.0/24
158.69.241.0/24
158.69.243.0/24
158.69.26.0/24
158.69.53.0/24
158.69.54.0/24
167.114.100.0/24
167.114.101.0/24
167.114.103.0/24
167.114.116.0/24
167.114.157.0/24
167.114.158.0/24
167.114.159.0/24
167.114.173.0/24
167.114.209.0/24
167.114.210.0/24
167.114.211.0/24
167.114.64.0/24
188.165.243.0/24
192.95.29.0/24
192.99.10.0/24
192.99.100.0/24
192.99.101.0/24
192.99.13.0/24
192.99.14.0/24
192.99.15.0/24
192.99.160.0/24
192.99.175.0/24
192.99.18.0/24
192.99.236.0/24
192.99.36.0/24
192.99.37.0/24
192.99.4.0/24
192.99.5.0/24
192.99.6.0/24
192.99.7.0/24
192.99.9.0/24
198.27.64.0/24
5.39.20.0/24
51.222.43.0/24
51.81.24.0/24
51.81.98.0/24
51.83.140.0/24
51.89.203.0/24
54.38.31.0/24
91.134.248.0/24

2001:41d0:1:0000:0000:0000:0000:0000/64
2001:41d0:1004:0000:0000:0000:0000:0000/64
2001:41d0:2:0000:0000:0000:0000:0000/64
2001:41d0:303:0000:0000:0000:0000:0000/64
2001:41d0:8:0000:0000:0000:0000:0000/64
2001:41d0:a:0000:0000:0000:0000:0000/64
2604:2dc0:100:0000:0000:0000:0000:0000/64
2607:5300:120:0000:0000:0000:0000:0000/64
2607:5300:203:0000:0000:0000:0000:0000/64
2607:5300:60:0000:0000:0000:0000:0000/64

[The Shower Scene]

Be tidy, organize your banned IPs numerically. It helps in the long run. The banning never stops.

5.39.0.0/17
5.135.0.0/16
5.196.0.0/16
8.18.128.0/24
8.18.172.0/24
8.20.110.0/24
8.21.41.0/24
23.92.224.0/19
37.187.0.0/16
37.60.48.0/21
37.60.56.0/21
46.105.0.0/16
46.105.198.0/24
46.105.199.0/24
46.105.200.0/24
46.105.201.0/24
46.105.202.0/24
46.105.203.0/24
46.105.204.0/24
46.244.32.0/20
51.38.0.0/16
51.68.0.0/16
51.75.0.0/16
51.77.0.0/16
51.79.0.0/17
51.79.128.0/17
51.81.0.0/17
51.81.128.0/17
51.83.0.0/16
51.89.0.0/16
51.91.0.0/16
51.161.0.0/17
51.161.128.0/17
51.178.0.0/16
51.195.0.0/16
51.210.0.0/16
51.222.0.0/16
51.254.0.0/15
54.36.0.0/16
54.37.0.0/16
54.38.0.0/16
54.39.0.0/16
66.70.128.0/17
79.137.0.0/17
92.222.0.0/16
103.5.12.0/22
107.189.64.0/18
135.125.0.0/17
135.125.128.0/17
135.148.0.0/17
137.74.0.0/16
139.99.0.0/17
139.99.128.0/17
141.94.0.0/15
142.4.192.0/19
142.44.128.0/17
144.2.32.0/19
144.217.0.0/16
145.239.0.0/16
146.59.0.0/17
147.135.0.0/17
147.135.128.0/17
149.56.0.0/16
149.202.0.0/16
151.80.0.0/16
152.228.128.0/17
158.69.0.0/16
164.132.0.0/16
167.114.0.0/17
167.114.128.0/18
167.114.192.0/19
167.114.224.0/19
176.31.0.0/16
178.32.0.0/15
185.12.32.0/23
185.45.160.0/22
185.228.96.0/24
185.228.97.0/24
185.228.98.0/24
185.228.99.0/24
188.165.0.0/16
192.95.0.0/18
192.99.0.0/16
192.240.152.0/21
193.70.0.0/17
193.104.19.0/24
193.109.63.0/24
195.110.30.0/23
195.246.232.0/23
198.27.64.0/18
198.27.92.0/24
198.50.128.0/17
198.100.144.0/20
198.245.48.0/20
205.218.49.0/24
213.32.0.0/17
213.186.32.0/19
213.251.128.0/18
216.32.192.0/24
217.182.0.0/16

[not2easy]

Hi JamieW and welcome to WebmasterWorld [webmasterworld.com]

As mentioned, it makes it easier to check against other lists if they are arranged in numeric order. There may be dozens of shared lists around here and it always helps to be able to update our info. As you've noticed, OVH and Hetzner are some major players. Amazon has outdone them all I believe, and then there is DigitalOcean, Leaseweb and so many others that add to our workload. It can help to determine whether your latest visit is a residential customer if you are looking at header requests.

The place to report new finds is in the Server Farms threads - current version: [webmasterworld.com...]

You can check for the habits of various UAs in the UA threads and in lucy24's "At Home With the Robots" series. The list is found here: [webmasterworld.com...]

Search and you might find more than you wanted...

[Brett_Tabke]

many of the bots are using headless Chromium these days. That means you can drop a redirect in on them.

Say for example, if someone pulls 10k views and they are not a clean google or bing ip, then redirect them to Google and let Google ban them after they pull down too many requests.

I know this works, because I was contacted by a bot runner who couldn't figure out how it was done...

[SumGuy]

I have a custom-written windows app that loads my various IP blocking lists and lets me easily test IP's and CIDR's to see if they're in my lists.

Of the OP's list of 100 CIDR's, 95 of them are in my www blocking list (which contains over 61k CIDR's). The 5 that are not are:

213.32.0.0/17
213.186.32.0/19
213.251.128.0/18
216.32.192.0/24 (CenturyLink)
217.182.0.0/16

Ironically, looking at today's logs I see reason to add 213.32.0.0/17 (smtp attempts).

All of those are OVH except for Centurylink. I'm puzzled by Centurylink, I don't know if they're an ISP or Hoster/server farm, but I have seen quite a bit of head-scratching web hits from them over the years. OVH is probably the one outfit that I haven't done a full IP-block on, but instead have blocked large CIDR's over time as I've seen them hit my servers. Their AS16276 has 131 CIDR's (4 million IP's) which can be boiled down to 92 contiguous CIDR's, of which I see I'm not blocking 7 of them. It could be they've added those recently.

[Jonesy]

Over on my VPS I have OVH blocked - originally by ASN assignments, then by CIDRs they rent from others.
Still on my VPS, I've always seen A LOT of Charter Communication. Recently many more Comcast customers seem to have been cracked.

[martinibuster]

Many ISPs (including Charter and others) force their combo router/modems on their customers, with some of that equipment having vulnerabilities on them. Also, many modems have similar problems. Then there are the many IOT devices that lack any kind of security.

I too have seen automated spam originating from ISPs, like Verizon, which was really weird. I suspect it was a botnetted computer.