Problems with IP Blocking?

Need to block some nasties, don't want to deny real visitors though

[MatthewHSE]

I tried searching for this topic, but all the terms I can think of are so generic that the results are pretty much useless...

Anyway, the problem is that I have a script that is getting requested quite a bit by robots. The script was previously (though briefly) exploited with e-mail form injection techniques, and although I plugged the holes right away, the script is still getting all kinds of requests.

Obviously I'd like to block these requests, and I'd just use IP banning to do that - except that many of the IP's making the requests are from the U.S., and I don't want to block them if there's any possibility of blocking legitimate U.S. visitors at the same time.

At this point, I don't even know what my questions should be. Basically I need to know what problems (if any) exist with blocking IP addresses, what I should consider when choosing which IP's to block, and that sort of thing.

Incidentally, I am logging the requests complete with the POST data so I can definitely pinpoint which requests are attempted hacks. It's just that I don't want to block any real visitors at the same time I block the bots.

Thanks in advance,

Matthew

[lammert]

The basic problem with blocking IP's is that you might block an IP which happens to be a proxy for many users, for example from AOL. If the IP's are coming from the US, you might try the WHOIS search at www.arin.net [arin.net]. The result of your search will be a page with information about the registered block of IP's that the bad behaving IP is in. With this information you can decide for yourself if blocking it will have the risk of blocking a large group of users.

Another test is to try a reverse DNS lookup with the command-line program nslookup which is available on most operating systems. If there is no reverse DNS entry, it is probably a single user connection. If it is a proxy or other larger system, the reverse DNS entry is most of the time present and quite informative.

[MatthewHSE]

Okay, thanks, that helps clarify things a lot. Unfortunately it will be a lot of time and effort to check all of these IP's, as they're coming in pretty fast and often. (Maybe 200 per day.) Is there any kind of tool I can use to check a long list of IP's all at once?

[Staffa]

As a short term solution, while you sort out the IP blocking, can you not rename your script and move it to another directory. Then bots will get a 404 not found and move on.

[Mokita]

I've dealt with this problem by putting all forms into a folder of their own, then using an .htaccess file in that folder to ban all requests that have no referer and no user-agent, which is most common for those bot requests.

It works like a charm.

[MatthewHSE]

Thanks for the suggestions folks. I did rename the script for over a week, but the bots never quit requesting it. That flooded my server with a ton of 404 requests. So I put up a dummy script with the same name which logs the IP and request data to a file which I can use to ban the IP's later, assuming I decide to go that route.