Welcome to WebmasterWorld Guest from 34.201.121.213

Forum Moderators: mack

Message Too Old, No Replies

Bing Publishes Bingbot Mobile Agents and Latest Developments

     
3:28 pm on Nov 12, 2014 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month Best Post Of The Month

joined:May 9, 2000
posts:26186
votes: 968


Bing has published an introduction to its mobile bots, along with the latest developments.

As you may know from our help topic Which Crawlers Does Bing Use [bing.com], we have a number of crawlers to perform our common crawl duties. To understand how your site behaves specifically for our mobile searchers, we have added a couple of new crawler variants which identify themselves with a user agent that mimics some of the most common mobile device types. In general, these crawlers use a user agent string that follow the following format:

Mozilla/5.0 + (Mobile Device) + Mobile Engine + Mobile Browser + bingbot/BingPreview/[version] Bing Publishes Bingbot Mobile Agent, and its Latest Developments [blogs.bing.com]


Here are a couple of example user agent strings that follow this format:

Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 BingPreview/1.0b

Mozilla/5.0 (Windows Phone 8.1; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 530) like Gecko BingPreview/1.0b

Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)

Mozilla/5.0 (Windows Phone 8.1; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 5
7:14 am on Nov 28, 2014 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15705
votes: 813


Woo hoo, look what I found in logs, hot off the presses:

157.55.39.39 - - [25/Nov/2014:18:35:09 -0800] "GET /ebooks/paston/paston4.html HTTP/1.1" 200 299510 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0;  http://www.bing.com/bingbot.htm)" 
157.55.39.207 - - [25/Nov/2014:18:35:28 -0800] "GET /ebooks/paston/pastonstyles.css HTTP/1.1" 200 10746 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 7_0 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11A465 Safari/9537.53 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"


These two were its very first appearance; there were some more the next day.

Note this detail:
bingbot/2.0;  http://www.bing.com/bingbot.htm

vs.
bingbot/2.0; +http://www.bing.com/bingbot.htm

That's apparently not a fluke. The version with the + sign is used for supporting files; the version with two spaces is used for pages.

:: wandering off to tweak Ignore criteria in log-wrangling code ::
9:11 am on Nov 28, 2014 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month Best Post Of The Month

joined:May 9, 2000
posts:26186
votes: 968


Yes, it's on a crawl (not a roll lol) right now.