GlobalMax - 12:10 pm on Sep 16, 2011 (gmt 0)

Can you use pattern matching to cut down on the number of entries in your robots.txt file?

The patterns that Google supports in robots.txt are described here [google.com ]. (Look under "Pattern matching," after expanding the "Manually create a robots.txt file.")