Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Google's Current Specifications for Robots Directives
g1smd - 8:33 am on Nov 25, 2010 (gmt 0)
Useful document, clarifies a lot of things.
Needs very careful reading.
Thread source:: http://www.webmasterworld.com/robots_txt/4234719.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com