Forum Moderators: open
What I've been doing is adding almost anything that requests robots.txt to the directive. I've started to think that this might not be such a good idea. Can you guys think of any time I wouldn't want to disable sessions for a UA that has requested robots.txt in the past? That is, other than what is obviously just someone typing in :
www.mysite.com/robots.txt
Thanks!