Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- Why Google Might "Ignore" a robots.txt Disallow Rule


jdMorgan - 12:18 am on Aug 16, 2006 (gmt 0)


I would recommend going with the Standard, and coding to the lowest common denominator to avoid problems. While our esteemed Googlers will undoubtedly provide authoritative answers for how Google does things, and Google's implementations are usually correct, compliant, and robust, extending those attributes to 'most robots' is rather a big stretch.

Jim


Thread source:: http://www.webmasterworld.com/google/3044757.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com