homepage Welcome to WebmasterWorld Guest from 54.227.20.250
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Microsoft / Bing Search Engine News
Forum Library, Charter, Moderators: mack

Bing Search Engine News Forum

    
msnbot
whats the use?
sidyadav

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 21 posted 7:24 am on Mar 18, 2004 (gmt 0)

Why does msnbot re-index thousands of web pages a day, when the results go nowhere? whats the use? I mean, if it was launching in May or June (2004), it would be a logical thing, but lets face it, rumors say its gonna launch in 2005 or even 2006, then why is it wasting so much of our bandwidth now?

What I have done is currently banned the spider via robots.txt. However, I will remove the ban once the real MSN search launches - in a year or two.

Life's too short to waste MB's of bandwidth for no reason :)

Sid
[note: by re-indexing, I did not mean indexing. Its logical for it index data slowly and to re-index it once in a while (once a month?) but totally wrong for it to re-index it in every day or two.]

 

sem4u

WebmasterWorld Senior Member sem4u us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 21 posted 8:15 am on Mar 18, 2004 (gmt 0)

Maybe there is a lot of testing going on behind the scenes? Maybe they are testing out how to make a fresher index than Google? Maybe they will launch sooner than you think?

sidyadav

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 21 posted 8:32 am on Mar 18, 2004 (gmt 0)

yeah, maybe

Sid

sem4u

WebmasterWorld Senior Member sem4u us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 21 posted 8:35 am on Mar 18, 2004 (gmt 0)

No-one knows Sid - we just have to wait and see :)

Chico_Loco

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 21 posted 6:45 am on Mar 27, 2004 (gmt 0)

Why bother banning the bot?

Surely it can't be taking all that much bandwidth?

sidyadav

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 21 posted 8:14 am on Mar 27, 2004 (gmt 0)

yes, it does not take much bandwidth in a day, but the thing is, it comes everyday to my site, downloads the whole site and leaves. Think long-term. How much bandwidth will it waste, everyday, until it launches in what? 1 year? 2 years?

Waste of bandwidth is much more valued than waste of time. ;)

Sid

Chris_D

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 21 posted 11:00 am on Mar 27, 2004 (gmt 0)

Sid - I'm with you.

MS = sizzle.

I'm waiting for the steak. Your site is huge Sid - block 'em. See if they actually respect robots.txt

I reckon if its not in msword format - thye'll ignore it anyway!

:)

mikep

10+ Year Member



 
Msg#: 21 posted 11:22 am on Mar 27, 2004 (gmt 0)

MSN search may become THE most used search engine in the future. I wouldn't ban msnbot. At some time it will stop reindexing your site everyday.

sidyadav

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 21 posted 11:55 am on Mar 27, 2004 (gmt 0)

MSN search may become THE most used search engine in the future. I wouldn't ban msnbot. At some time it will stop reindexing your site everyday.

In my original most, I said:
However, I will remove the ban once the real MSN search launches - in a year or two.

I reckon if its not in msword format - thye'll ignore it anyway!

lol, if they don't obey no robots.txt, I'm gonna ban them through .htaccess or IP - thats for sure.

I also see that WebmasterWorld has blocked msnbot, I guess Brett's been witnessing the same situation I am.

Sid

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Microsoft / Bing Search Engine News
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved