WebJoe - 10:12 pm on Jan 6, 2004 (gmt 0)

In my case, google is just showing the url of the page, no title, no snippet...but the page is indexed. So technically, it does follow the robots.txt and does not visit the page. But it indexes it, since it's linked to from a page that's allowed by robots.txt.

But my intention by disallowing it was to have the page not being indexed. Hence my second question,

Do I have to use the robots-metatag to prevent these pages from being indexed?

@KenB: I do use this as my bad bots trap, I have one directory mentioned in robots.txt with a default page that will add the user agent string and IP to a database as a source for banning visitors.