Bewenched - 5:08 am on Dec 1, 2012 (gmt 0)

You can then use logs to catch scrapers since they'll be the ones hitting up your pages fast and furious

Sometimes they do it slowly too though.

I spend about an hour every night looking over the log from the day. I end up blocking about 2-3 user agents a night. It's run rampant especially out of the Ukraine, which has really forced us to block the country and a lot of russia. Shame we do get a few customers a year from Russia. And dont even get me started about China. You have to be careful blocking there since some of the ranges actually support Australia too.

You would think that having webmaster tools and giving the pages you want crawled and the date of modification that somehow google would take that into account.