goodroi - 1:11 pm on Dec 31, 2006 (gmt 0)

I do agree with Jim that it would be ideal to make a robots.txt and allow everything.

Damon also makes a good point in keeping it simple since I have helped many people who had their sites fall out of the search engines because they made a badly formatted robots.txt. The webmasters never used a validator to verify the robots.txt was correct. This is not to say robots.txt is hard. It is more a story of not being a lazy webmaster.