How long to ban an IP?

[jay5r]

I'm just curious if there are best practices for how long to ban a misbehaving IP. Here are a few scenarios - let me know your thoughts on each…

• A bot that's not abiding by your robots.txt
  
• A bot that's posting spam
  
• A bot that's actively trying to do fairly serious harm (SQL injection, etc.)
  
• A person (validated by reCAPTCHA) who is posting spam
  
• A person you've banned from a social-type site who may may be upset enough by the ban that they'd try to harm the site
  

Would you ban for a longer time if the IP has a history of problems? If so, how far do you look back, or how what type of historic patterns would make you give a longer ban?

Would you ban for a longer period of time if the IP is listed in AbuseIPDB?

Do you give longer bans if the IP is from a country that tends to have more problem traffic than good traffic?

And related to all that - what are your thresholds for banning an entire subnet?

Thanks in advance for your feedback!

[lucy24]

Are you thinking of specific, individual, down-to-the-last-digit IP bans, or entire blocks? Wider ranges are easier, because most of them are colos or server farms that can safely be banned forever with no harm done. I keep a separate list for temporary bans--either individual addresses, or ranges representing human ISPs from countries that aren't in my target demographic and seem to be more vulnerable to infection. I check the temporaries after six months or so; if there aren’t any recent requests it is generally safe to assume that either the originating computer has been disinfected, or the offending robot has moved.

[jay5r]

Re: "Are you thinking of specific, individual, down-to-the-last-digit IP bans, or entire blocks?" - I currently look at the list of problem IPs and ban subnets based on how many in the subnet are known to be problem IPs.

4+ out of 16 (/28 or /60)
6+ out of 32 (/27)
10+ out of 64 (/26)
18+ out of 128 (/25)
32+ out of 256 (/24 or /56)

Beyond that I just ban specific IPs (or the first 4 segments for IPv6).

At the moment I don't really consider the nature of their hosting - just the nature of their actions.

[not2easy]

When a bot is coming from a datacenter or server farm IP, there is not much value to leave doors open and close a window. If you simply use whois to lookup IPs you may save a ton of unnecessary work by closing the doors and windows.

This topic is discussed in the Search Engine Spider and User Agent Identification [webmasterworld.com] forum. There's a handy menu of several years of shared experiences here: [webmasterworld.com...] or you can search for known offenders and IP ranges.

Rarely I've seen bots coming from apparent residential IPs and most of those 'ISP' IPs also offer hosting services.

[jay5r]

On my to-do list is to process postback pings (when a transaction is completed), which would would come from data centers. So blocking all data centers would make that impossible. I'd have to figure out which ones will be sending postback pings and make sure not to block them.

But I can see blocking data centers in places like China.

Also - if I blocked a data center would it then interfere with me initiating a connection to a server in that data center? My server does pull assets from my sponsors' servers.

[not2easy]

Access logs can provide the specifics, I would not suggest that anyone simply block ranges without determining where the unwanted traffic originated. The same data can provide you with the specifics of your 'wanted' traffic.

[lucy24]

if I blocked a data center would it then interfere with me initiating a connection to a server in that data center?

Not unless you’ve got a setup that requires the other server to make a POST or PUT request to your own server. Ordinarily, access controls apply to HTTP requests from wherever-it-is, not requests from your own server to wherever.

And you can always poke holes. Details will of course depend on your server type, but the general idea is
BLOCK all of 1.2.3
EXCEPT specifically 1.2.3.4

If you're getting unwanted requests from the identical IP that you're using for wanted content, you've got more problems than this forum can handle :) (Or, possibly, your advertisers are making routine requests that have been misinterpreted as unwanted robotic activity.)

[jay5r]

Thanks for the replies - but no one other than Lucy really addressed the question of what variables determine how long you block an IP address for - other than a few people suggesting that traffic from data centers can be blocked pretty liberally.