Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Sky IP requesting all files alphabetically

cyberdyne - 3:24 pm on May 7, 2011 (gmt 0)


Has anyone else noticed this?
IP 2.220.166.34

Requests all files in the home directory in alphabetical order but doesn't stick to any robots rules.

Thread source:: http://www.webmasterworld.com/robots_txt/4309286.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com