Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Amazon AWS Hosts Bad Bots


Pfui - 11:46 pm on Oct 25, 2011 (gmt 0)


Got twitter-swarmed a bit ago. In addition to a boatload of AWS bots, two particularly bad ones:

ec2-184-73-108-194.compute-1.amazonaws.com
MetaURI API/2.0 +metauri.com
robots.txt? NO
ERROR: Client sent malformed Host header <-- x2

ec2-50-18-24-18.us-west-1.compute.amazonaws.com
percbotspider
robots.txt? NO
ERROR: Client sent malformed Host header <-- x2


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4368965.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com