Msnbot

To ban or not to ban - that is the question

[PhraSEOlogy]

Msnbot is hitting my site constantly and taking 1000's of pages. That in itself is not a bad thing - but here is the rub.

Google crawls and takes pages - a few days later they are in the index - NO problem.

Msnbot crawls and crawls and then puts 1 page (index.html) in its index - when I know it has many 1,000s of pages - thats a waste of bandwidth.

Does anyone know if msn will use any of the pages it crawls (for free) or are they going to be dumped after testing?

[bears5122]

I wouldn't ban it. Microsoft is going to be a huge player in the search engine world and I wouldn't mess with it. You could be kicking yourself a year or two from now if your rankings in MSN aren't as good as they could have been.

[digitalv]

I agree with bears... If Microsoft's track record says anything about how they like to do things, MSN search is probably going to be bigger than Google whether you like it or not. That's not a search engine I want to exclude myself from :)

[PhraSEOlogy]

Thanks Guys.

With the recent drop in google traffic over the last two days there is NO WAY I am going to ban msnbot now. If my google traffic goes kaput then I can wait till MSN gets the new search online.