Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Amazon AWS Hosts Bad Bots


dstiles - 7:46 pm on Apr 5, 2012 (gmt 0)


I generally block IPs using a custom-built trapping system, since I use IIS.

If an IP range gets really troublesome I add the range (or part of it) to the IIS Directory Security restrictions list. I have had to do this with a few amazon ranges.

If you have htaccess (linux/unix and some IIS servers) then use that.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4368965.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com