Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Why robots.txt if you want everything indexed?

OntheEdge - 2:22 pm on Mar 3, 2003 (gmt 0)


>>You could just put an empty robots.txt file in there.<<

Does that help?
What scenerio could you see with no robot.txt at all?
I've read a lot about keeping bots out, but know little about using robot.txt to draw them in deeper.

Thread source:: http://www.webmasterworld.com/robots_txt/33.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com