Welcome to WebmasterWorld Guest from 54.147.189.54

Forum Moderators: mack

Message Too Old, No Replies

msnbot/1.0 crawling again?

Ok, so how may does msn have crawling anyway?

     
4:15 pm on Feb 1, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 27, 2002
posts:1685
votes: 0


65.54.188.50 - - [31/Jan/2005:03:54:22 -0800] "GET /robots.txt HTTP/1.0" 200 1751 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"

Curious mind wants to know.

8:48 pm on Feb 1, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Feb 24, 2004
posts:219
votes: 0


Yes, it does. It visited my site 43 times last night.
9:09 pm on Feb 1, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:May 14, 2002
posts:378
votes: 0


Spotted here 1/24.
IP:
207.68.146.*
207.46.98.*
65.54.188.*
9:13 pm on Feb 1, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 27, 2002
posts:1685
votes: 0


I don't get it.

Unless MSN is going to assign different tasks to different bots, what advantage does reverting to an older version serve?

12:56 am on Feb 2, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member steveb is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:June 20, 2002
posts:4652
votes: 0


They've crawled a lot, but their "starting" index ended up not very good in terms of depth of domain, as is evidenced by many threads here. Experimenting with older or other bots makes sense if they are trying to fix that deficiency.
1:33 am on Feb 2, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 27, 2002
posts:1685
votes: 0


Thanks steveb, makes sense to me. :)
1:55 am on Feb 2, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 8, 2004
posts:102
votes: 0


Could it possibly be a bot for the beta in other countries? There could be an iterative process for the beta bots.