Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Noindex vs Disallow within a Sitemap


phranque - 11:19 pm on Jul 26, 2012 (gmt 0)


how the bleepity bleep do you prevent g### from indexing your sitemap and robots.txt?


you could always try using the X-Robots-Tag HTTP header:
http://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag [developers.google.com]


Thread source:: http://www.webmasterworld.com/robots_txt/4478700.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com