Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Bots ignoring my robots.txt

roddy - 7:10 am on Nov 5, 2003 (gmt 0)


Thanks for that. Actually the last 24 hours seem to have seen Googlebot calm down and pay attention to the robots.txt. I'll need to wait a while to be really sure, and if I have any more problems I'll try your suggestion.

Thanks for all the help

Roddy

Thread source:: http://www.webmasterworld.com/robots_txt/184.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com