Welcome to WebmasterWorld Guest from 126.96.36.199 , register , login , search , subscribe , help , library , PubCon , announcements , recent posts , open posts Subscribe to WebmasterWorld
msnbot/1.0 crawling again? Ok, so how may does msn have crawling anyway? pendanticist msg:1532439 4:15 pm on Feb 1, 2005 (gmt 0)
188.8.131.52 - - [31/Jan/2005:03:54:22 -0800] "GET /robots.txt HTTP/1.0" 200 1751 "-" "msnbot/ 1.0 (+http://search.msn.com/msnbot.htm)"
Curious mind wants to know.
flashfan msg:1532440 8:48 pm on Feb 1, 2005 (gmt 0)
Yes, it does. It visited my site 43 times last night. fiestagirl msg:1532441 9:09 pm on Feb 1, 2005 (gmt 0)
Spotted here 1/24. IP: 207.68.146.* 207.46.98.* 65.54.188.* pendanticist msg:1532442 9:13 pm on Feb 1, 2005 (gmt 0)
I don't get it.
Unless MSN is going to assign different tasks to different bots, what advantage does reverting to an older version serve?
steveb msg:1532443 12:56 am on Feb 2, 2005 (gmt 0)
They've crawled a lot, but their "starting" index ended up not very good in terms of depth of domain, as is evidenced by many threads here. Experimenting with older or other bots makes sense if they are trying to fix that deficiency. pendanticist msg:1532444 1:33 am on Feb 2, 2005 (gmt 0)
Thanks steveb, makes sense to me. :) houseofsecrets msg:1532445 1:55 am on Feb 2, 2005 (gmt 0)
Could it possibly be a bot for the beta in other countries? There could be an iterative process for the beta bots.