agerhart - 8:52 pm on Jun 18, 2002 (gmt 0)

I think I may have not clarified what I meant, and I think it comes down to the way that you like to set yours up.

I always list out all of the important spiders and robots in my robots.txt and then specify if they have full access, partial access, or no access at all.

In my opinion, this makes it easier to change it in the future.