Welcome to WebmasterWorld Guest from 54.145.85.22

Forum Moderators: Ocean10000 & incrediBILL & keyplyr

Message Too Old, No Replies

bingbot joins the 131.253 party

     
9:30 pm on Sep 1, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13210
votes: 347


There are two recent threads about msnbot-media visiting from the 131.253.41. range:

[webmasterworld.com...] (1 month ago)

[webmasterworld.com...] (2 months ago)

I've had them set to Ignore all along, because they're well-behaved on my site. (ymmv, as shown in those earlier threads). Yesterday for the first time I met the ordinary bingbot from the same neighborhood:

131.253.47.251 - - [31/Aug/2012:21:33:15 -0700] "GET /robots.txt HTTP/1.1" 200 911 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
131.253.47.251 - - [31/Aug/2012:22:36:07 -0700] "GET /fun/AlonzoMelissa.html HTTP/1.1" 200 10423 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"


That's .47. while all the mediabots have used .41. in the same range. But again, unmistakable bingbot behavior, with more requests for robots.txt than for pages.

Huh.

D'you suppose it's got something to do with the spike in bingbot activity reported elsewhere? 157.55. got too crowded so they had to outsource?

:: minor detour here to cross-check search and discover that amazon has the brazen gall to label something as "copyrighted material" although it is scraped from Project Gutenberg and therefore by definition in the public domain ::
9:38 pm on Sept 1, 2012 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14650
votes: 94


It returns the proper rDNS msnbot-131-253-47-251.search.msn.com
8:06 pm on Sept 2, 2012 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3121
votes: 3


Lucy, you missed my posting here... [webmasterworld.com...]

It gives all the ranges (approx) that I could get from DNS.
3:13 am on Sept 3, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13210
votes: 347


Today I got a swathe of bingbots from that range

Yup, I did miss that. Best excuse: you were heroically trying to stay OT while the rest of the thread got
:: cough-cough ::
a bit sidetracked.

Philosophically interesting, though. Robot A has never offended me, while Robot B is firmly in Shoot to Kill territory. The next guy comes along and reports the exact opposite experience.

And, uhm, yes, I did mistype the topic header. S/b 131.253 of course, as in the body of all relevant posts. That would be the range Microsoft is subletting from Avaya, not to be confused with the range Amazon is subletting from Merck.
5:40 am on Sept 3, 2012 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14650
votes: 94


Has anyone done a reverse DNS scan thru all those IP ranges to catalog which IPs claim to be specifically for bingbot?
8:48 pm on Sept 3, 2012 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3121
votes: 3


The 131.253 range? Yes. I ran a DNS scan 29th Aug. The ranges I posted elsewhere are fairly accurate - possibly about 20% IPs not used for bot (that's "intelligent guesswork" without actually checking).