webdudek - 9:10 am on May 21, 2008 (gmt 0)
I find this discussion amusing.
There wasn't anyone around here asking the question - "Why?"
Why do you care that your robots.txt is in google index?
What is the possible damage?
What could go wrong if someone reads it?
I have many pages (robots, sitemaps, disclaimers, privacy policies, login forms) on my sites that google had indexed and are completely useless to anyone. I really couldn't care less.
if someone wants to read my robots.txt - be my guest!
Anyone has a good reason for this discussion?