homepage Welcome to WebmasterWorld Guest from 54.227.56.174
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
msnbot/1.0 from CN
real or fake?
thetrasher




msg:3124397
 5:48 pm on Oct 17, 2006 (gmt 0)

219.142.53.11 and 219.142.53.17
(no ptr records)
UA "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
no referer
HTTP/1.0
reads robots.txt and default main page

IP belongs to CHINANET beijing province network

 

fiestagirl




msg:3126110
 7:24 pm on Oct 18, 2006 (gmt 0)

Unfortunately no answers but the same question.

Saw something similar from:
219.141.219.***

Same UA and no ptr.

Ocean10000




msg:3126135
 8:01 pm on Oct 18, 2006 (gmt 0)

I have started seeing MSNBOT/1.0 returning a hostname like the following as of this last Monday.

207.46.98.73 -->
livebot-207-46-98-73.search.live.com -->
207.46.98.73

Which seems to be following Googles lead by using dns lookups as a way to identify the real bot verses someone trying to spoof them.

This is only for MSNBOT/1.0 so far not the MSNBOT-media and other variants from Microsoft.

MSNBOT include an extra Header "From" with a value "msnbot(at)microsoft.com" which can also help weed out spoofers.

Ocean10000




msg:3133873
 2:05 am on Oct 25, 2006 (gmt 0)

I just seen this on one of my sites on this ip 219.142.53.21
everything else was identical.

Only checked Robots.txt and left (which it was baned) so it seems to be well behaved.

wilderness




msg:3134335
 1:48 pm on Oct 25, 2006 (gmt 0)

Although not entirely on-topic my inquiry for the same bot.

I seem to recall that 1.0 was the MS image bot?

As previously mentioned in another thread
( [webmasterworld.com...] ), I have severe problems of communication with the many MS bots.

Yesterday, I have 1.0 crawling with the same UA from three entirely different ARIN ranges.

65.54.188.149
64.4.8.136 (denied)
207.46.98.111

previous days
202.96.51.149 (APNIC denied)

msn-dude informed us a while back that MS was sorting out their bot identities and UA's however these inconsistecies seem to just add more confusion.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved