Microsoft and 13.64.0.0/11

Legit bing-bots? BingLocalSearch?

[SumGuy]

Does Microsoft (aka Bing) operate any "legit" search bots in the 13.64.0.0/11 block? Such as these:

13.66.139.x (Bingbot)
13.82.29.x (BingLocalSearch)
13.90.209.x (BingLocalSearch)
13.92.250.x (BingLocalSearch)

I also see something called a semantic bot in this range. I have a grand total of 116 hits since Jan 2015 from this /11 CIDR, and I see that I have it largely blocked. Any downside to this? Today I have opened 13.66.0.0/16 but I'd still like to know what the consensus is for this /11 - and what is BingLocalSearch?

[not2easy]

I see them from 13.66.139.x with UA "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)". That's not in depth research, just glanced at what I was checking over the past week.

MSFT has that block and a few others nearby:
13.64.0.0 - 13.107.255.255
13.64.0.0/11, 13.96.0.0/13, 13.104.0.0/14

re: what is BingLocalSearch? I know nothing.

[lucy24]

Does Microsoft (aka Bing) operate any "legit" search bots in the 13.64.0.0/11 block?

Sure. 13 is another of those 8/s we were recently talking about in another thread. In this case, it formerly belonged to Xerox but, just to be different, they haven't sold all of it to AWS. 13.64-107 (i.e. 64-95, 96-103 and 104-107, go figure) is now Microsoft.

Microsoft/bing has never had as narrowly delimited a crawl range as G###. If it looks, quacks and otherwise acts like the bingbot, and comes from a range that belongs to some aspect of Microsoft, it's generally safe to assume it's legit.

[SumGuy]

Starting a few months ago we started to do port-80 blocking in the router (based on a big list) and 13.0.0.0/8 was in the list. Incoming packets get dropped, so they don't even get a 403. Something in 13.66. 139.x was hitting us a few days ago, like 10 times over maybe a few hours, and for some reason the last one got through to our web server. The server has it's own list, which also did include the 13/8 block, so it generated a 403, and that's what tweaked me on this. I had a closer look at the logs (from before that /8 was blocked) and discovered some AWS hits from other parts of 13/8, as well as these MS/Bing hits. I had no idea bing was doing stuff from there. So I'm still blocking the entire /8 except for 13.66.0.0/16. Still want to know what the binglocalsearch is.

[lucy24]

I took a quick detour to archived logs. I don't see a lot of bingbot visits from 13, but enough that I can't call it a fluke. Almost all 13.66, most of those 13.66.139 with a scattering of ..205 and ..207. But I also find a tiny handful of 13.77, so keep an eye out for those too. 13.64.0.0/12 would cover it. Each of the 13.77s, incidentally, was immediately adjacent to other bingbot visits from other known crawl ranges. That's a typical bingbot behavior.

[iamlost]

When it comes to SE crawler identification I simply take each SE at their word.

Bing states that a reverse DNS lookup will return a name ending in search.msn.com. After which a forward DNS lookup must return original IP address.

If for whatever reason a SE breaks their own crawler ID rule I consider their problem.

And as I am unable to find any official info on a BingLocalSearch it either is not a bingbot or Bing is breaking - strike 2 in this instance - their own rule to their detriment at my sites.

But then I allow very few bots. Your requirements may differ.

Note: I actually do very well in regard to Bing referred traffic so my tight door policy apparently isn't hurting.

[lucy24]

When it comes to SE crawler identification I simply take each SE at their word.

I hope you don't apply that to all the robots calling themselves Googlebot.

[topr8]

... he means you should be able to do a reverse and forward dns lookup to check it is legit - as Microsoft say that's what should be! (don't google say the same about same with googlebot)

[dstiles]

I have a few subranges blocked in that /12 for semanticbot. Are those still turning up from there? (Not saying they use all of the IPs noted below but some, at least.)

13.82.56.0 - 13.82.63.255
13.90.208.0 - 13.90.215.255
13.92.136.0 - 13.92.143.255
13.92.232.0 - 13.92.255.255

[blend27]

31.*

I have an exact # of 962 requests from 13.66.139.0 <<<---- yes that IP(sorry) starting on 09/01/2018 on one of my sites that got blocked due to no RDNS round trip.

Usually for the pages that rank on a first page of Bing for Long Q.

RE: semantic bot had 2 UAs......
semanticbot(13.82.62.**)
semanticbot ([e-mail]) that came from 13.92.25.**

[dstiles]

Thanks for the semantic ones, blend27. I have neither of them, but the ranges are blocked anyway. I tend to filter known unwanted bots through a faster filter.

[Steven29]

I dont think so. There is microsoft cloud servers too. Bing webmasters tools has a tool ylthat allows you to validate the bing bot. This should help determine if its legitimate.

[blend27]

-- Bing webmasters tools --

I go down 4 flights of stairs to pick up my irregular paper mail, and then same 4 up. I have an agreement with Mail Lady, she is awesome except for the days she brings actual bills, that she rings the bell when the mail is here...

or one should not be able to install Win7/10+ without a proper key unless it is cracked.

Logging into BING WMT later on is no deal here.

.. the point... Round trip DNS must match the IP.

Goes both ways.

[blend27]

As far as 13.66.139.0 I mentioned before: I was simply blocking it via my software firewall, but since the start and till now it has hit over 400 times I just moved the rule into web.config where all requests from this IP will be simply dropped. I used [bing.com...] tool to verify that IP.

I got this from there:

Verdict for IP address 13.66.139.0:
No - this IP address is NOT a verified Bingbot IP address.