Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
keyplyr - 12:08 am on Nov 15, 2012 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4518888.htm
Bewenched could you supply more info? Did it ask for robots.txt? If so did it obey? What type files did it retrieve? How many?
Brought to you by WebmasterWorld: http://www.webmasterworld.com