Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Bots ignoring my robots.txt
Nick_W - 12:06 pm on Nov 3, 2003 (gmt 0)
Are the pages it's requesting dyname? - like privmsg.php?x=y&a=b? I think bots see that as a different page.
Nick
Thread source:: http://www.webmasterworld.com/robots_txt/184.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com