Robert_Charlton - 8:08 am on Aug 3, 2010 (gmt 0)

pontifex - Very thoughtful post... thanks for sharing your observations.

...Google will return the blocked URLs (and how many of them there are) when you use the site:- operator

It's possible for a competitor to find out a lot about you by using the site: operator, which is another reason not to use robots.txt to block pages you don't want referenced. Use meta robots noindex if you don't want URLs referenced.