signor_john - 9:21 pm on Dec 19, 2009 (gmt 0)

The fact I don't lock my door when I go out (ie. don't disallow robots to cache my content) doesn't give a thief a good explanation in court when he walks in and steals from my house.

That's a questionable analogy, because search engines merely index content. The analogy would work only if a search engine ignored robots.txt or an intranet firewall (like a burglar ignoring your door lock), took everything off your servers, and left no files behind.