Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Amazon AWS Hosts Bad Bots

Pfui - 8:33 pm on Nov 12, 2011 (gmt 0)


Yet another AWS somethingorother:

ec2-175-41-250-151.ap-northeast-1.compute.amazonaws.com
ceron.jp/1.0

robots.txt? NO

[robtex.com...]

Thread source:: http://www.webmasterworld.com/search_engine_spiders/4368965.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com