- - Search Engines
- -- Sitemaps, Meta Data, and robots.txt
- ---- Why should I have a robots.txt file?
DamonHD - 5:04 pm on Dec 30, 2006 (gmt 0)
My feeling is, in the spirit of KISS (Keep It Simple, Stupid), that unless you NEED it, don't put it up at all, and filter out any (spurious) errors from its absense in other ways.
Thread source: http://www.webmasterworld.com/robots_txt/3203372.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com