msnbot and bandwidth

[digtoo]

So I did a site search and see some other complaints about the msnbot and bandwidth usage. Here's my story for this month alone:

- msnbot 1.01 GB used so far this month
- googlebot 509.72 MB used so far this month

- so far 7638 links from google searches
- so far 39 links from msn searches

Yeah, I know MSN is planning on a new release, but 1 GB, nearly double that used by Google, is too much, especially for something forthcoming! The speed of requests isn't an issue for me, but the amount of bandwidth is definitely an issue.

What gives? I am this close to banning msnbot.

[diamondgrl]

When you say 39 links from msn searches, are you looking at msn.com or the prototype site? If you are looking at the former, that's Yahoo's search engine. Therefore, do not confuse it with msn.bot.

I personally would not ban a major search engine such as msnbot. But then again, I don't really know what your hosting situation is and your cost of maintaining this hypothetical future revenue.

[Robino]

Hi Digtoo.

Go ahead, take the risk and ban it.

But seriously, why gamble? 1.01GB- BIG DEAL!

Maybe this is MSN's way of weeding out Mickey Mouse sites.

Personally, I think banning it is a bad long-term move.

[digtoo]

When you say 39 links from msn searches, are you looking at msn.com or the prototype site?

I'm looking at the AWStats output. There is a separate listing for MSN and for Yahoo, but I do not know if that means they are different.

But seriously, why gamble? 1.01GB- BIG DEAL!

I guess bandwidth usage, as with a lot of things, is relative per person. IMHO, I don't want to get stuck paying bandwidth overages.

[diamondgrl]

digtoo,

i think your referals from msn mean the msn that uses the yahoo search engine. if so, banning the msnbot would be rash since this may provide a decent fraction of your referalls very soon (i don't follow closely enough to know if msn has announced a launch date but earlier reports said it would be this year).

unless you have more specific reason, stick with it.

[isitreal]

the msn bot is grabbing much more than the googlebot currently, and much faster. Personally, what worries me is the snail like pace googlebot travels as it runs through new pages on sites, not how fast MSN bot munches our sites. Check out the msn beta preview, links on another thread here, see how your site does, you might find that it's not a bad idea letting msn work on your site, we don't know when msn is going live, but the new beta stuff looks a lot better than the last version.