dstiles - 7:38 pm on Apr 26, 2013 (gmt 0)
I am reasonably happy with my own server host but I also block their complete range of server IPs.
There is no reason for one server to access another outside of a mutually established relationship. Search engines are tolerated - just - but 99.99% of all bots are harmful or wasteful or both.
If a company offers broadband then let them identify this fact in their DNS records; few do, although some ISPs are now starting to add DSL to their DNS entries. If they operate only server farms or are suspect and do not identify broadband ranges they get blocked. Being the best server host on the planet has nothing to do with blocking their IP ranges.
It is my own experience that hurricane electric users have only my disinterest at heart, no matter how good the service itself may be; although I admit they are by no means the worst offender. But unless they can be shown to have broadband ranges they are completely blocked. Why would you NOT block a server farm, excepting only specific beneficial bots? I block most of G and MS, excepting only certain bot ranges: I certainly would not hesitate to block arbitrary scrapers and (quite probably) compromised servers.
I have, in the past, posted replies in the google forum hereabouts to people complaining their sites are being scraped. I suggested they employ bot-blocking methods to alleviate their problems. Not even a reply, just more complaints about scraped content!