Play_Bach - 1:51 pm on Nov 25, 2005 (gmt 0)

Getting back to "rogue bots" as Brett's stated reason for having to shut down search, I keep coming back to the question:

"If rogue robots are such a problem, then how do the big sites deal with it?"

Somehow they all must, or they'd all be out of business, right? So how come WebmasterWorld can't?