Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Google's Current Specifications for Robots Directives
tangor - 5:52 am on Nov 25, 2010 (gmt 0)Thread source:: http://www.webmasterworld.com/robots_txt/4234719.htm
"robots.txt"? We don't need no steenkin' robots.txt
Except for those that do honor whitelisting, etc. Google, as usual, seeks to bend standards to their standard. And, sad to say, the sheeple webmasters will follow.
Brought to you by WebmasterWorld: http://www.webmasterworld.com