Page is a not externally linkable
keeper - 2:26 am on Sep 21, 2003 (gmt 0)
From this behaviour I would take the leap and say that: At least for Google, not sure how the other engines will take it. I checked the robots documentation as well and couldn't find any specific examples..
Following on from your Google example,
A direct request to:
/mac
Shows this URL has both pagerank and cache info.
However:
/groups
(which is blocked in the robots.txt) has neither.
/this.php will be crawled and indexed if your robots.txt has a directive to disallow: /this.php?