Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Robots.txt standard
ssn5054 - 6:40 am on Dec 7, 2001 (gmt 0)Thread source:: http://www.webmasterworld.com/robots_txt/124.htm
I would go with the first one too. The robots file is checked by the spider for anything that pertains to it, all other permissions are ignored.
Brought to you by WebmasterWorld: http://www.webmasterworld.com