Welcome to WebmasterWorld Guest from 220.127.116.11
Forum Moderators: mack
I believe it's an experimental project which they plan to launch few months down the line, which could easily postpone to much more time than that.
How often will MSNBot access a page from my web server?
In general MSNBot should not try to access your site more than once every few seconds. MSNBot will also account for the time it takes to download a page from a site so tht if your site has a slower connection we will not access it as frequently. If you find that we are placing too high a load on your site please let us know by sending us e-mail at email@example.com.
I wonder what similarities there are between sites w/ the difficulty. Perhaps something makes MSNBot think there are more pages than there are. Here's an idea ...
If page.asp?item=1234&last=5432 has a link to page.asp?item=8897&last=1234 etc. ... then a nearly infinite combination of URL's would exist.
Are your sites set up anything like that?
I put the following lines in my robots.txt file:
And I even banned the IP address it seems to be coming from in my .htaccess file:
deny from 65.54.164.
What else can I do? It's hogging up my bandwidth and I don't care to be in Microsoft's directory. (I hate Microsoft in general) The majority of the world uses Google and people can find trommetter.com that way.
Anyone know how to slow him down?
Absolutely - MSN built a special robots command "delay" into their msnbot for this purpose - launched in this very forum a couple of weeks ago :)