Brett_Tabke - 3:10 pm on Dec 27, 2002 (gmt 0)

1- what possible use could an image bot do for us other than use bandwidth?

2- some bots do support robots.txt and do use standard agents. It catches a few.

3- off topic referrals and local related discussions.

4- I spend an hour to two hours a day tracking down bad bots and banning them, their ip, or there entire isp. It isn't bad when you have a few dozen or few hundred pages; but, when you get into 70-80k pages, bots can tear a system up in no time flat without sufficient tracking and response systems. It is a very serious problem that threatens the entire system on a daily basis. We are having a discussion about moving to a subscription based system and stopping rogue bots would be my #1 reason for doing so.