bingdude - 12:15 am on Feb 1, 2012 (gmt 0)
Well, your current record is 102 (one hundred two) consecutive successful requests for robots.txt. This seems a bit over the top :) Is it bing's subtle way of telling me that they don't like the robots.txt and keep hoping it will change? Is bingbot offended because it's lumped under the generic "Hey you!" of "User-Agent: *" instead of getting its very own set of rules?
I hope you meant "discrete". The idea of a discreet URL does make an interesting mental picture, though.
Quick run through the logs confirms that it isn't having any trouble finding pages. Including :: cough, cough :: a fair number of explicit "index.html" that sneaked into various internal links. It's just the infatuation with robots.txt that makes it all look so skewed.
First off, let's raise a glass here. Lucy posted the 17,000th post in this forum! :)
Now, this could also be an indication that we like your site. ;) Bingbot does follow the robots.txt directives, which means it does access the file when it visits the website. So... frequent visiting could see us accessing the file frequently... You want we should move along...? ;) (Kidding!)
I'd be concerned if the robots.txt file was the ONLY file we checked out on the site and if we stopped indexing pages. That would be an issue.