Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- 80legs Flooding Site - Ignoring Robots.txt
Andem - 11:40 am on May 30, 2012 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4457359.htm
Just wanted to note in this thread that contacting them by email does seem to work. They didn't tell me who it was that wanted all of our data, but they did promise not to spider us anymore. Since then, they haven't visited our site.
Brought to you by WebmasterWorld: http://www.webmasterworld.com