homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

librabot/1.0 ( http://search.msn.com/msnbot.htm)
Something new from MSFT

 4:55 am on Dec 21, 2008 (gmt 0)

librabot/1.0 ( [search.msn.com...]
inetnum: -
netname: Microsoft
country: CN
descr: 6 Buildings Xigema Center No.49 Zhichun Road,Haidian District, Beijing,P.R.China

Took robots.txt and default root file then went away.

Has anyone seen this before? I see lots of log file entries, but no explanation what it does. Thanks.



 3:44 pm on Dec 22, 2008 (gmt 0)

just seen the first tiem too: - - [22/Dec/2008:16:25:33 +0100] "GET /robots.txt HTTP/1.0" 200 34 "-" "librabot/1.0 (+http://search.msn.com/msnbot.htm)" - - [22/Dec/2008:16:25:34 +0100] "GET /some-content-site.html HTTP/1.0" 200 29712 "-" "librabot/1.0 (+http://search.msn.com/msnbot.htm)"


 3:22 am on Jan 2, 2009 (gmt 0)

Here's some startling MSNBOT news:


 3:30 am on Jan 27, 2009 (gmt 0)

I have also seen this bot. Resolves to China....MSN bot from China?
Traces to: CNCGROUP Beijing Province Network / Microsft (China) Co.Ltd

The one that hit my website, also from China and did not even ask for a robots.txt file....how rude! librabot/1.0 (+http://search.msn.com/msnbot.htm)
traces to: Data Communication Division / Beijing Telecom

Now I dont know about all you guys, but I have a long history of...shall we say "undesirable traffic" that emanates from China. Until I see a consistant "user agent" and traceable IP range for this bot, its getting blocked!

[edited by: Lain_se at 3:33 am (utc) on Jan. 27, 2009]


 11:16 pm on Jan 28, 2009 (gmt 0)

Microsoft says to block librabot



 12:09 am on Jan 29, 2009 (gmt 0)

Welcome to WebmasterWorld.

What I'm seeing is Brett Yount, Moderator and Program Manager, Live Search Webmaster Tools, recommends blocking an IP Address in China. Depending on your traffic that may or may not be a good idea. All he's really saying about it is blocking that IP Address will not block msnbot. :)


 12:26 am on Jan 29, 2009 (gmt 0)

"My traffic" is 65% botnet spam agents and malicious content scrapers. If Microsoft can't identify some of this new traffic then I block it and get on with my day.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved