Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- BeetleBot
keyplyr - 12:08 am on Nov 15, 2012 (gmt 0)
Bewenched could you supply more info? Did it ask for robots.txt? If so did it obey? What type files did it retrieve? How many?
Thread source:: http://www.webmasterworld.com/search_engine_spiders/4518888.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com