bingbot joins the 131.253 party

[lucy24]

There are two recent threads about msnbot-media visiting from the 131.253.41. range:

[webmasterworld.com...] (1 month ago)

[webmasterworld.com...] (2 months ago)

I've had them set to Ignore all along, because they're well-behaved on my site. (ymmv, as shown in those earlier threads). Yesterday for the first time I met the ordinary bingbot from the same neighborhood:

131.253.47.251 - - [31/Aug/2012:21:33:15 -0700] "GET /robots.txt HTTP/1.1" 200 911 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
131.253.47.251 - - [31/Aug/2012:22:36:07 -0700] "GET /fun/AlonzoMelissa.html HTTP/1.1" 200 10423 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

That's .47. while all the mediabots have used .41. in the same range. But again, unmistakable bingbot behavior, with more requests for robots.txt than for pages.

Huh.

D'you suppose it's got something to do with the spike in bingbot activity reported elsewhere? 157.55. got too crowded so they had to outsource?

:: minor detour here to cross-check search and discover that amazon has the brazen gall to label something as "copyrighted material" although it is scraped from Project Gutenberg and therefore by definition in the public domain ::

[incrediBILL]

It returns the proper rDNS msnbot-131-253-47-251.search.msn.com

[dstiles]

Lucy, you missed my posting here... [webmasterworld.com...]

It gives all the ranges (approx) that I could get from DNS.

[lucy24]

Today I got a swathe of bingbots from that range

Yup, I did miss that. Best excuse: you were heroically trying to stay OT while the rest of the thread got
:: cough-cough ::
a bit sidetracked.

Philosophically interesting, though. Robot A has never offended me, while Robot B is firmly in Shoot to Kill territory. The next guy comes along and reports the exact opposite experience.

And, uhm, yes, I did mistype the topic header. S/b 131.253 of course, as in the body of all relevant posts. That would be the range Microsoft is subletting from Avaya, not to be confused with the range Amazon is subletting from Merck.

[incrediBILL]

Has anyone done a reverse DNS scan thru all those IP ranges to catalog which IPs claim to be specifically for bingbot?

[dstiles]

The 131.253 range? Yes. I ran a DNS scan 29th Aug. The ranges I posted elsewhere are fairly accurate - possibly about 20% IPs not used for bot (that's "intelligent guesswork" without actually checking).