Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- importance of a robots.txt file


Daily_Sparring - 3:47 pm on May 20, 2004 (gmt 0)


Thanks JD, one other question then, do you link to that text file from within the head tags? I am assuming so otherwise, how would the spider no to look for the robot file first? Again, sorry for my lack of knowledge here.


Thread source:: http://www.webmasterworld.com/robots_txt/384.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com