Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
creative_craig - 3:54 pm on May 7, 2003 (gmt 0)Thread source:: http://www.webmasterworld.com/robots_txt/18.htm
I have put up a file on my site called robots.txt telling all spiders that they can spider, it keeps the 404's down.
Brought to you by WebmasterWorld: http://www.webmasterworld.com