homepage Welcome to WebmasterWorld Guest from 23.20.44.136
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Bing user agent?
Does Bing use Msnbot or not?
ning




msg:3924419
 5:24 am on Jun 2, 2009 (gmt 0)

Is the new engine Bing using Msnbot, or does it have its own spider identification?

 

badbadmonkey




msg:3924881
 4:39 pm on Jun 2, 2009 (gmt 0)

Live? BingBot? I'd like to know this also - anyone noticed anything different in their logs?

Pfui




msg:3925204
 1:47 am on Jun 3, 2009 (gmt 0)

You may need to be registered/logged in if you want to access this link... From the "Your website disappears from Bing results [help.live.com]" section, emphasis mine:

"The web crawler used by Bing is also known as MSNBot..."

I wonder which one(s)? Here are just some of MSN's identifiable bots; these have 'msnbot' in the UA:

msnbot/1.0 (+http://search.msn.com/msnbot.htm)
msnbot/1.1 (+http://search.msn.com/msnbot.htm)
msnbot/2.0b
msnbot-media/1.0 (+http://search.msn.com/msnbot.htm)
msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)
MSNBOT_Mobile MSMOBOT Mozilla/2.0 (compatible; MSIE 4.02; Windows CE; Default)
msnbot-Products
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; livebot-searchsense/0.1; +http://search.msn.com/msnbot.htm)

-----
Here's a good how-to/overview about Bing's Webmaster Tools from Sascha Kimmel:

How To Use The Bing Webmaster Tools To Get Info On Your Site [saschakimmel.com]

-----
And finally, a brief report aprés my first visit to Bing.com, using the same login as Live. In a nutshell? MORE problems than with Live. Dangit.

1.) Top-rated site's info line above the link is truncated to two names; nothing about the site proper; nothing from the home page's title.

2.) Variation on same site's name shows zero results, despite being equally findable/visible on Google.

3.) Authentication shows as failed for the two sites I'd added to Live. Both sites authenticated A-OK for as long as I can remember; one via META tag, the other via XML file. There's no change in the on-site tag/file: Each is properly set up with the same codes provided (by Live and now Bing).

Misc. Bits:

- The only MSN bot I've ever blocked from accessing other than robots.txt is the incredibly misbehaved msnbot/2.0b. From the looks of the above snafus, apparently some algo somewhere determined it's not nice to block bad MSN bots...

- Heck, those two sites are practically nonexistent in Bing's SERPS now; about on a par with the traffic they've ever received from Live, etc. Might finally be the right time to bid adieu to MSN/Live/Bing entirely, particularly after years of accommodating MSN's messed-up bots and/or attempting to fix MSN's messed-up SERPS.

LOL. An obsolete meaning for the word "bing" is "to go." More current parlance (Bing cherries notwithstanding) is... "a heap or pile."

You be the judge.

: )

[edited by: Pfui at 1:49 am (utc) on June 3, 2009]

badbadmonkey




msg:3925214
 2:18 am on Jun 3, 2009 (gmt 0)

I've noticed that a particular article I have on my site with a "no-cache" tag is not ranking at all in Bing. It's listed, it comes up on a listing of the site pages, but doesn't rank at all for any of the keywords it should. Other pages linking to it make an appearance instead.

Other articles, all structured the same way, rank highly as they should.

It seems that Bing is penalizing the no-cache attribute? (Although it does respect it, the page isn't cached).

Pfui




msg:3925407
 7:40 am on Jun 3, 2009 (gmt 0)

1.) If there's any newfound favoritism toward caching, that'd be the last straw for me. I've had to run the MSN request-content-removal gamut three times this year already to remove caches they've saved/made available despite no-cache meta tags and customized Cache-Control headers in .htaccess.

However, even I'd be reluctant to leap to any penalty-related conclusions until multiple others confirmed same.

2.) One more "msnbot" --

Not new, but blocked since initial Live.com authorization because real-people UAs don't include the word 'webmaster.' If anyone else finds their site(s) suddenly failing authorization, see if you're denying --

msnbot-webmaster/1.0 (+http://search.msn.com/msnbot.htm)

Hailed from: msnbot-207-46-126-nn.msn.com
Requested: root dir; and LiveSearchSiteAuth.xml

[edited by: Pfui at 7:45 am (utc) on June 3, 2009]

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved