Page is a not externally linkable
g1smd - 11:23 pm on Apr 26, 2011 (gmt 0)
Googlebot crawls them now when listed in robots.txt. They don't index, but they crawl. I've never been fond of robots.txt because Google interprets the guidelines literally. I've seen sites show thousands, hundreds of thousands of URI only entries due to this crap.
I see no evidence that Google crawls these URLs (i.e. fetches these from the server). They simply add the URL to their database and list it as a URL-only entry in the SERPs.
If you are certain the application does not generate abnormal links then you're ok because you don't care what others are injecting or if external sites manipulate your site's links.
The only way to get dupe content is if your domain somehow generates or recreates the duplicated links or if it's prone to URL poisoning.
You have made this bold statement in several recent threads. It isn't true. If Google requests a URL and it returns "200 OK" then it is fair game for indexing.