Welcome to WebmasterWorld Guest from 23.20.241.155

Forum Moderators: mack

Message Too Old, No Replies

msnbot/1.0 crawling again?

Ok, so how may does msn have crawling anyway?

     
4:15 pm on Feb 1, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



65.54.188.50 - - [31/Jan/2005:03:54:22 -0800] "GET /robots.txt HTTP/1.0" 200 1751 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"

Curious mind wants to know.

8:48 pm on Feb 1, 2005 (gmt 0)

10+ Year Member



Yes, it does. It visited my site 43 times last night.
9:09 pm on Feb 1, 2005 (gmt 0)

10+ Year Member



Spotted here 1/24.
IP:
207.68.146.*
207.46.98.*
65.54.188.*
9:13 pm on Feb 1, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I don't get it.

Unless MSN is going to assign different tasks to different bots, what advantage does reverting to an older version serve?

12:56 am on Feb 2, 2005 (gmt 0)

WebmasterWorld Senior Member steveb is a WebmasterWorld Top Contributor of All Time 10+ Year Member



They've crawled a lot, but their "starting" index ended up not very good in terms of depth of domain, as is evidenced by many threads here. Experimenting with older or other bots makes sense if they are trying to fix that deficiency.
1:33 am on Feb 2, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks steveb, makes sense to me. :)
1:55 am on Feb 2, 2005 (gmt 0)

10+ Year Member



Could it possibly be a bot for the beta in other countries? There could be an iterative process for the beta bots.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month