Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Noindex vs Disallow within a Sitemap


lucy24 - 7:17 pm on Jul 25, 2012 (gmt 0)


Short answer: the word 'noindex' is not part of the Robots Exclusion Standard. Use it at your own risk.

Disallow = robots stay out, no crawling allowed
Noindex = page is not mentioned in google's* search index

Yes, a page can be indexed even if a search engine has not seen it. They only have to know it exists.


* I say specifically google, because That Other Search Engine has indexed a few pages that are clearly and explicitly labeled noindex.


Thread source:: http://www.webmasterworld.com/robots_txt/4478700.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com