- - Search Engines
- -- Sitemaps, Meta Data, and robots.txt
- ---- Bots ignoring my robots.txt
Nick_W - 12:06 pm on Nov 3, 2003 (gmt 0)
Are the pages it's requesting dyname? - like privmsg.php?x=y&a=b?
I think bots see that as a different page.
Thread source: http://www.webmasterworld.com/robots_txt/184.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com