Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- -very- much the same robot


keyplyr - 5:46 pm on Sep 22, 2012 (gmt 0)


It may not "done anything to cause serious annoyance" in your logs but what is it doing with the data it mines from your site? And what do the guys that purchase your data do with it?


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4498453.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com