-- Search Engine Spider and User Agent Identification
---- Filtering Out Really Hard To Find Bad Bots
wilderness - 2:20 pm on Jan 19, 2013 (gmt 0)
Those IPs are handled on an individual basis because you couldn't block all of Comcast or Cox unless you wanted to lose a ton of customers.
Bill, I agree with the broad statement of denying an entire provider, however if a webmaster is aware of the origin of traffic for his site (s), than he could certainly deny a sub-net region, or even a portion of a region. EX: I get very little valid traffic from the San Francisco Bay Area, regardless of provider.
Visitors from Nebraska, The Dakota's Oklahoma, Colorado, Las Vegas, Oregon, Washington, in fact for me most everything west of the Mississippi. These folks are simply without interests in widgets.