my site is getting hammered, what can I do?

looking for pointers on how to block bots and all that fun stuff...

[Craig_F]

one of my sites is loved by all bots apparently, I'm on shared hosting now and had to shut my site down temporarily since I was killing the server. Oops! :)

while I investigate my options (dediated servers, etc.) I figured I should start looking into what I can do to reduce all this activity WITHOUT harming my real traffic.

so, I know there's bad bot lists and that you can do things like slow down the MSN bot spidering, but where do I start? Meaning I know of these things, but I have no clue how to use them properly, anyone care to share or point me to a thread that covers what I should do?

Thanks!

[The Contractor]

Use .htaccess.

Here are [google.com] "some" threads on the subject

[Craig_F]

thanks for the pointer. now I'm looking at using the list from posts in msg#3 and msg#9 in this thread:

[webmasterworld.com...]

Anything better/more I can do?

Also, once this is up and running any ideas what level of improvement I'll see? I realize there's no way to tell from one site to the next, but generally does this type of thing help a lot or a little?

[The Contractor]

does this type of thing help a lot or a little?

Well it stops anything you put in there (.htaccess)dead in it's tracks. If it's not in there... it can't stop it. So it will do only as good as you maintain it ;)