Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- importance of a robots.txt file
sem4u - 3:49 pm on May 20, 2004 (gmt 0)Thread source:: http://www.webmasterworld.com/robots_txt/384.htm
Just put the robots.txt file in your root directory. There is no need to link to it anywhere. It is usually the first file that spiders read when accessing your site.
Brought to you by WebmasterWorld: http://www.webmasterworld.com