List of Data Centre IP ranges for blocking purposes

[Xpat]

My need is to block VPN services due to continued abuse of my forum website. Obviously the vast majority of these requests come from data centres. I've tried to find a list of known data centres and their IP ranges but all I can come up with is IncrediBILL's block posts on the subject, and the following hint that such a list exists for those 'in the know' . . .

[googleonlinesecurity.blogspot.com...]

I simply don't have the resources to compile this list myself. Is there such a list in existence, and is there a list that is actively maintained?

[wilderness]

No such copy and paste solution exists.
A search on google for VPN provides an approximate 102-million results.

FWIW, the bots and other mischievous IP's and/or tools that penetrate blogs/forums (regardless of venue, whether WP, SMF or other OS) are quite different, and more numerous than any general widget website (they just keep coming).
Additionally, and given the increase of Cloud services by mobile devices your objective is even more difficult and time consuming.

[not2easy]

You're better off to investigate your raw access logs to see which particular IPs are abusing your site. Blocking all of them is inefficient and quite an additional workload for your server that could slow down loading. It's a balance of shutting the door on the most abusive IPs and keeping an eye out for new ones.

They don't stand still, new server farm IPs are reported in these forums nearly every day - another reason not to rely on someone's list. It is static, likely outdated and may not do anything for the ones hitting your site. If you can't do it yourself you may find a list around the net that may or may not be current and might address some of the IPs you need to block. It is like fixing a leaking roof with a blanket. I recommend against looking for a list. It can't do what you hope to get done.

[Xpat]

OK thanks for the commentary. Amazon (for example), which is an unflushed toilet of scraper and VPN excrement, willingly publish their IP ranges and I sure blocked those. That really made a significant dent in abuse. I just wish I had access to the lists 'the industry' has compiled.

[keyplyr]

@Xpat - Be aware that if you block server ranges you will also block humans. That's just a fact. You will never know how many humans you are blocking until you do the work.

For example, the highly vilified Amazon sent me over 2 thousand humans yesterday... and yes, I block all Amazon ranges BUT with conditions. Through a series of checks (IP, header, UA, behavior, etc) I allow the humans (and some non-humans) access. This applies for ALL server ranges, and I block thousands of them.

If you are not willing to manually inspect your daily server logs and do the research, don't block IP ranges since you will never really know what is coming from them. Thousands of mobile apps that bring humans to our sites make their connections through cloud servers (Amazon, etc) There are also hundreds of proxy ranges that bring humans within server farm ranges. There are also schools, gov't agencies, social sites & other companies that may be a huge benefit to your site.

You learn this by manually inspecting your daily logs, researching, writing the code & testing. Never, ever cut'n paste a block list from someone else.

Blocking all of them is inefficient and quite an additional workload for your server that could slow down loading.

I disagree about slowing down loading. Done properly, it makes absolutely no impact on page loads. My current htaccess is approx 260KB with about 4 thousand ranges blocked, over 200 rewrites & a couple dozen other lines. The only thing in there that has any impact at all are the half-dozen redirects. Google says my pages load faster than most (or whatever their exact wording is.)

However, I used to think that as well. That's what we were all told 10 or 15 years ago. Possibly it had a bigger impact on pages loads in the past when servers were slower. But there are many articles that disprove that myth today and I have proof :)

[tangor]

Read through the last year's worth of posts in this Spider Forum for some of the more readily identifiable (and routinely known to be undesirable) server farms, colo's, or ip ranges. Also make the determination if country denial would be of benefit to you.

If you are in sales/products/ecommerce, losing one human might be a potential customer. If you are anything else, you just lost a visitor who may or may not have brought you value. If you are a publisher, you do want human eyeballs on the page. You will have to make those determinations.

Some of my sites are locked down (country, farms, colo's, ip ranges) to refine the traffic desired. Others are more open with any denials set are for abuse of bandwidth (bot traffic for example). As others have noted above, this is an on going maintenance struggle. There is no cut and paste list that will solve your problems... and many of those lists might be outdated in that some once bad locations have become valued locations.

Your raw access logs are your definitive information on these things. If you don't know how to read them, or use them, then learn. It will make a big difference in many ways (better traffic, controlling bandwidth costs, avoiding threats to your site).

[investedOnes1]

I have a similar problem but mine relates to abuse from Amazon AWS. I have an IP address blocker but it doesn't prevent the culprits from coming back. Reporting it to Amazon's abuse department seems fruitless. Has anyone successfully blocked Amazon IP addresses?

[not2easy]

If you have an IP blocker in effect and it does not serve 403s to IPs you have blocked then you should examine the rules rather than add more IPs. It might help to spend some time reading in the Forum Library: [webmasterworld.com...] for this (and the Apache) Forum, then ask if you need some fine tuning help.