Brett_Tabke - 9:54 pm on Nov 27, 2005 (gmt 0)

> Ban the IP for an hour or so

There are some bot runners that think it is ok to run bots if they are respectful and at a slow pull rate. They think a page view per half minute is ok. Then they leave them on for the entire week and forget about them. An hour ban would stop maybe a third of the bots. This isn't an average site architecture or visitor. These are highly knowledgeable, highly motivated, and proactive visitors who are not new to site ripping.

Then we get into the hyper, super, merciless monitoring services. There must be 100 of these things running from 500 IPs. These guys are so totally out of control it isn't funny. I had calls this weekend asking why we had banned their xyz robot and they threatened us if we did not allow them back in. [webmasterworld.com...]