Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- importance of a robots.txt file

sem4u - 3:49 pm on May 20, 2004 (gmt 0)


Just put the robots.txt file in your root directory. There is no need to link to it anywhere. It is usually the first file that spiders read when accessing your site.

Thread source:: http://www.webmasterworld.com/robots_txt/384.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com