- - Search Engines
- -- Sitemaps, Meta Data, and robots.txt
- ---- Sky IP requesting all files alphabetically
cyberdyne - 3:42 pm on May 7, 2011 (gmt 0)
Yes I have a site map. I had a feeling it was something like that. Have blocked them for now, just to dissuade them.
Thread source: http://www.webmasterworld.com/robots_txt/4309286.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com