enigma1 - 8:00 pm on Apr 12, 2012 (gmt 0)

The bot knows of the existence of the URL, but (theoretically) not the contents.

That only applies if the bot access the robots.txt and then crawls some pages. But it doesn't apply when the bot gets redirected or finds a link to that page from another site, in which case it may access it directly. At least in the past when I tested it the bot did access the contents. If they would follow the robots.txt strictly I would expect them to access it before every page access since that's another file that can change at anytime.