g1smd - 6:44 pm on May 16, 2008 (gmt 0)

I needed a robots.txt file to be gone from the index.

I added this to robots.txt:

Disallow: /robo

Google picked up the new robots.txt file after a few days (for use in checking what they can index).

They looked at it several times each week, but the OLD robots.txt content remained in the SERPs and in their cache and snippet for many weeks.

About a month later, the robots.txt file dropped from the SERPs.

You can use robots.txt to block the indexer from indexing its content.

That will not restrict access for the bot that retrieves the file for its real intended purpose.