homepage Welcome to WebmasterWorld Guest from 23.22.217.122
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
librabot/1.0 ( http://search.msn.com/msnbot.htm)
Something new from MSFT
GaryK




msg:3812128
 4:55 am on Dec 21, 2008 (gmt 0)

librabot/1.0 ( [search.msn.com...]
219.142.53.23
No PTR
-----
inetnum: 219.142.53.0 - 219.142.53.127
netname: Microsoft
country: CN
descr: 6 Buildings Xigema Center No.49 Zhichun Road,Haidian District, Beijing,P.R.China

Took robots.txt and default root file then went away.

Has anyone seen this before? I see lots of log file entries, but no explanation what it does. Thanks.

 

xcomm




msg:3812793
 3:44 pm on Dec 22, 2008 (gmt 0)

just seen the first tiem too:

202.96.51.149 - - [22/Dec/2008:16:25:33 +0100] "GET /robots.txt HTTP/1.0" 200 34 "-" "librabot/1.0 (+http://search.msn.com/msnbot.htm)"
202.96.51.150 - - [22/Dec/2008:16:25:34 +0100] "GET /some-content-site.html HTTP/1.0" 200 29712 "-" "librabot/1.0 (+http://search.msn.com/msnbot.htm)"

incrediBILL




msg:3817620
 3:22 am on Jan 2, 2009 (gmt 0)

Here's some startling MSNBOT news:
[webmasterworld.com...]

Lain_se




msg:3835318
 3:30 am on Jan 27, 2009 (gmt 0)

I have also seen this bot.

202.96.51.149 Resolves to China....MSN bot from China?
Traces to: CNCGROUP Beijing Province Network / Microsft (China) Co.Ltd

The one that hit my website, also from China and did not even ask for a robots.txt file....how rude!

219.142.53.17 librabot/1.0 (+http://search.msn.com/msnbot.htm)
traces to: Data Communication Division / Beijing Telecom

Now I dont know about all you guys, but I have a long history of...shall we say "undesirable traffic" that emanates from China. Until I see a consistant "user agent" and traceable IP range for this bot, its getting blocked!

[edited by: Lain_se at 3:33 am (utc) on Jan. 27, 2009]

miqrogroove




msg:3836914
 11:16 pm on Jan 28, 2009 (gmt 0)

Microsoft says to block librabot

[forums.microsoft.com...]

GaryK




msg:3836937
 12:09 am on Jan 29, 2009 (gmt 0)

Welcome to WebmasterWorld.

What I'm seeing is Brett Yount, Moderator and Program Manager, Live Search Webmaster Tools, recommends blocking an IP Address in China. Depending on your traffic that may or may not be a good idea. All he's really saying about it is blocking that IP Address will not block msnbot. :)

miqrogroove




msg:3836951
 12:26 am on Jan 29, 2009 (gmt 0)

"My traffic" is 65% botnet spam agents and malicious content scrapers. If Microsoft can't identify some of this new traffic then I block it and get on with my day.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved