KenB - 7:03 pm on Jan 20, 2004 (gmt 0)
I simply created the entry into the robots.txt that stated: "disallow: /robots.htm", without actually creating the page and then look for 404 requests for this page in the log files. I don't do anything fancy. Since the page doesn't exist and a 404 error is generated, it will show up in my error report of my Analog reports. If I see the entry, I know I need to look at my logs to see what the bot was.