Page is a not externally linkable
gdawg - 2:59 pm on Jun 14, 2006 (gmt 0)
I have seen Google ignoring my robots.txt file as well. In my situation I have a page from my old site that I implemented a 301 redirect to the same page on my new site and excluded this page in my robots.txt file on the new site. For some reason Google is completely ignoring the robotx.txt file and this page is indexed and ranking on the 1st page of results.