kaled - 3:56 pm on Oct 13, 2010 (gmt 0)
This is not something I have studied, however, scraping seems to fall into two categories - Commercial Research and Adsense.
Google permits scraping because it doesn't care about intellectual property (except its own) and because they are now happy to be evil provided there's money in it and provided Joe Public doesn't catch on (that they are being evil).
So, to reduce the problem, either it must be made unprofitable to Google (that means court cases - good luck with that) or Joe Public must be educated as to what's going on and that it's all Google's fault (blame Bing as well if you like).
Copying data for any profitable purpose is likely to be a breach of copyright. A couple of test cases will be required to establish that is true even if the data is not republished. However, we'll still be in cival law territory - to really make a difference, a precedent would need to be set establishing that scraping is a breach of criminal law. Personally, I would think this is doable but I'm not a lawyer. However, international agreements would still be needed and that's not going to happen quickly.
It's hard to see any way to defeat scrapers altogether by blocking - even if you come up with the perfect piece of software, the potential currently exists to use botnets and defeating them will be real tricky. However, when the big boys are caught at it, naming and shaming might help.