Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- importance of a robots.txt file


caspita - 12:27 am on May 21, 2004 (gmt 0)


Thanks Jim,

So .. looks like it is also a good idea at the begining when your are trying to catch up the spiders .. telling them ..'hey .. here a new site .. you are welcome to spider me :-) '

Thanks again...
Carlos.


Thread source:: http://www.webmasterworld.com/robots_txt/384.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com