Forum Moderators: open

Message Too Old, No Replies

Microsoft UK

         

lucy24

5:50 pm on Oct 18, 2020 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



IP: 51.143.19.abc (mostly)
UA: standard bingbot UA with standard bingbot headers

I don't know if they are planning on making a habit of this. I first saw bingbot from 51.143 on 13 October, and more often in the last few days.

Tangentially, I didn't even realize 51.140-145 (i.e. 140/14 and 144/15) belonged to Microsoft. I've seen the range fairly often, but invariably used by malign robots who were blocked at the gate. This, on the other hand, shows every sign of being the legitimate bingbot: matching UA, matching headers--some of them fairly distinctive--matching habit of requesting nonexistent files thanks to getting their links garbled. No robots.txt requests as yet, but that isn't dispositive, since they're shared out among all their multifarious IPs.

It is, of course, possible that a maverick botrunner has hit on the trick of matching the bingbot's UA and headers, but there's nothing in the requests to raise suspicion. That is, no wp-admin, no xmlpr-whatsit and so on.

Hmm.

jmccormac

6:08 pm on Oct 18, 2020 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



No reverse dns as with genuine Bing crawlers. Given the incompetence of MSFT in the past, there could be a lack of joined up thinking and they did no set up the host names. Perhaps it might be worth pointing it out on Twitter? (Seen tens of thousands of requests that are broadly in sequence with genuine Bing requests it looks like Bing.)

Regards...jmcc

wilderness

9:28 am on Oct 19, 2020 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



lucy,
Have the entire 51 denied for an eternity.
One direct request on the 15th.
Four direct request on the 18th.
Nothing in my Sept logs.

lucy24

3:30 pm on Oct 19, 2020 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Have the entire 51 denied for an eternity.
I've got four specific /16 ranges within 15 denied, while most of 52 and 54 are bad_range (meaning they can be unset for distributed robots). Since I moved over to header-based access controls, IP lockouts are down to around 20 lines or so, most of them triggered by recurring malign robots with fully human headers.

Will headers be another of those things like "Mozilla" that used to be useful screens and will become less so over the years? Probably. But it hasn't happened yet.

jmccormac

5:39 am on Oct 26, 2020 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Seems that they could be genuine Bing crawlers. Have got crawl error messages from the Bing webmaster site.Not sure if they've added reverse DNS as they managed to get banned at IP level and they fail the Bing bot verification:
[bing.com...]

Regards...jmcc