- - Search Engines
- -- Sitemaps, Meta Data, and robots.txt
- ---- Does robots.txt override meta tag?
TallTroll - 3:26 pm on May 2, 2002 (gmt 0)
That file will lock out any spider that obeys the robots.txt. They shouldn't look at anything
Of course, not all spiders are that well behaved...
Thread source: http://www.webmasterworld.com/robots_txt/143.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com