Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Scooter


mack - 10:57 am on Jul 6, 2003 (gmt 0)


Scooter seams to respect robots.txt in my sites. Unusual for it to disrespect. Not being funny but are you sure you used the correct syntax in your robots.txt file?

Mack.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/2068.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com