Spider/site-grabber Trap

Is there really anything available.

[roycerus]

Okay so I have researched a bit and found the topics on www and other place regarding spider traps. But I recently read an article here about firefox pre-fetching links. Will it be reading the robots.txt before prefetching?

I read somewhere that there is a method to automatically ban IP addresses for a period of time when there are too many queries from a single IP. And then remove the IP after a certain period of time. Can this be done?

Is there any script which I can install which will handle these problems?

[roycerus]

Anyone has any idea?
I keep getting this error often asking for this file:
/_vti_bin/shtml.exe/_vti_rpc

USER:
Host --
IP -- 202.159.241.114
User --
Agent -- MSFrontPage/4.0
Cookies -- Array

Looks like there are often attempts to copy pages using frontpage. Can anyone help with any automatic htaccess modifying script?

[nancyb]

search for bad-bot [google.com] on Google

and this search for the string vti_bin/shtml.exe/_vti_rpc [google.com]

there are a number of previous posts about both :)