msr986 - 8:22 am on Feb 17, 2003 (gmt 0)

Wow, I'm not sure about this one.

There are two possibilities.

1. The bot will ignore the bad request and treat this like a 404 error. Since it cannot find or read the robots.txt, nothing is disallowed and the bot will proceed to request your regular pages.

2. The bot will recognize that the 401 error is caused by a page that requires authorization, and will treat this site as if it is off limits. Under this scenario, the bot will just leave.

I'm inclined to believe that the bot will react according to the first scenario.

The question is, did the bot grab any other pages besides the robots.txt? Check your logs.