- - Search Engines
- -- Sitemaps, Meta Data, and robots.txt
- ---- paging google! robots.txt being ignored!
Sanenet - 11:34 am on Nov 10, 2005 (gmt 0)
This is why we should all stick to a standard robotstxt standard (www.robotstxt.org) instead of each SE having it's own little rules! ([webmasterworld.com ]).
Thread source: http://www.webmasterworld.com/robots_txt/782.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com