homepage Welcome to WebmasterWorld Guest from 54.204.215.209
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor
Visit PubCon.com
Home / Forums Index / Search Engines / Alternative Search Engines
Forum Library, Charter, Moderators: bakedjake

Alternative Search Engines Forum

    
Hit by the Msnbot
Msnbot? Hmm, I thought it was a myth
akande




msg:464957
 2:45 am on Nov 2, 2003 (gmt 0)

My site has finally been crawled by the Msnbot. It gave [search.msn.com...] as the information URL.

It seems to be going quite slow, has anyone had their sites crawled by it yet?

 

pmac




msg:464958
 3:05 am on Nov 2, 2003 (gmt 0)

[webmasterworld.com...]

eatapeach




msg:464959
 7:43 am on Nov 7, 2003 (gmt 0)

i saw it show up for the first time around the middle of october. it didn't stick around long, i'm wondering what it was looking for :)

jeremy goodrich




msg:464960
 7:02 am on Nov 9, 2003 (gmt 0)

Finally saw it run through 1/3 of a site, seems that they are at least getting better at crawling the web...

pendanticist




msg:464961
 3:52 pm on Nov 28, 2003 (gmt 0)

"better" is much like 'beauty' these days '...in the eye of the beholder'.

:)

You may recall my domain was one of the first ones crawled w-a-a-a-y back when MSN's bots were but fledglings.

However, msnbot/0.11 makes only two requests. One for robots and one for the most obsure page of my entire domain.

It's been doing this every day since it started a week ago.

131.107.137.47 - - [26/Nov/2003:13:38:08 -0800] "GET /robots.txt HTTP/1.0" 200 1524 "-" "msnbot/0.11 (+http://search.msn.com/msnbot.htm)"
131.107.137.47 - - [26/Nov/2003:13:38:09 -0800] "GET /FileWhereI"Cite"Something.html HTTP/1.0" 200 1432 "-" "msnbot/0.11 (+http://search.msn.com/msnbot.htm)"

In my readings thru WebmasterWorld, I don't recall seeing anything specifically related to msnbot/0.11 and I'd like to get back to being crawled....regularly.

Anyone seeing similar?

I guess there is the off-chance that was all the bot was/is looking for.....

Pendanticist.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Alternative Search Engines
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved