Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Amazon AWS Hosts Bad Bots


grandma_genie - 8:09 pm on Jul 2, 2012 (gmt 0)


All these came in the other day, June 25, and all with the same UA, but different IPs, most are from Amazon. Now all blocked:

209.9.228.nnn - - "GET / HTTP/1.0" 301 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" Beyond The Network America, PullThePlug Technologies LLC
176.34.222.nn - - "GET /example HTTP/1.0" 301 249 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" Amazon Data Services Ireland Ltd
204.236.186.nnn - - "GET /example HTTP/1.0" 200 26468 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" us-west-1.compute.amazonaws.com
177.71.186.nn - - "GET /example HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" sa-east-1.compute.amazonaws.com Brazil
67.217.160.nn - - "GET /example HTTP/1.0" 301 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" Hostname: place.holder ISP Latisys-Ashburn, LLC Ashburn, VA
67.217.165.nn - - "GET /example HTTP/1.0" 200 22675 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" same as above
67.217.168.nnn - - "GET /example HTTP/1.0" 200 838 "mywebsiteurl" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" same as above
175.41.156.nn - - "GET /example HTTP/1.0" 200 22917 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" ap-southeast-1.compute.amazonaws.com Singapore
50.112.42.nn - - "GET /example HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" us-west-2.compute.amazonaws.com
50.57.168.nnn - - "GET /example HTTP/1.0" 200 28011 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" Hostname: static.cloud-ips.com Rackspace Hosting
184.73.73.nnn - - "GET /example HTTP/1.0" 403 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" compute-1.amazonaws.com Ashburn, VA
175.41.198.nnn - - "GET /example HTTP/1.0" 200 210463 "mywebsiteurl" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" ap-northeast-1.compute.amazonaws.com

I've listed behind the UA who they are and where they are from. What were they doing? Scraping...
--GG


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4368965.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com