Msg#: 4535380 posted 8:56 am on Jan 12, 2013 (gmt 0)
Yesterday got one from 220.127.116.11 for BingSiteAuth.xml, never had that file to begin with.
I usually get, a dozen or so requests a month, referrals from goog/bing for "add+exchange link" from IP ranges in India & Vietnam(which are 403d anyhow) followed by sitemap.xml requests from the same IPs.
Msg#: 4535380 posted 12:38 am on Jan 13, 2013 (gmt 0)
A solitary request by a standard users IP would NOT be a valid request.
Depends whether "standard user" means someone who has already set foot on the site and established themselves as human. Heck, I once got a request for "humans.txt" ;)
Yesterday got one from 18.104.22.168 for BingSiteAuth.xml
That's the standard Bing wmt file, like the google or yandex versions with all the numbers, from the now-standard IP for this request. I don't know when exactly they transferred this task to the 131.253 range. Somewhere between early June and early November, based on before-and-after around a big hole in my logs.
Much of the worry goes away when you add to htaccess something like
<FilesMatch "\.(js|txt|xml|php)$"> Header set X-Robots-Tag "noindex" </FilesMatch>
(adjusted for any non-page extensions you may happen to use) because really I'm more worried about google indexing my robots.txt than about malign robots reading it :)