-- Search Engine Spider and User Agent Identification
---- IP Bunch Analysis
incrediBILL - 7:42 am on Apr 6, 2012 (gmt 0)
Course everybody knows I'm an extremist.
Not when it comes to blocking China, the Great Firewall of China is far from extreme.
The Chinese have a bunch of startups all focused on crawling and/or scraping, the fancy name is aggregating. It's an epidemic, and blocking the entire country is the only vaccine. I also add Russia, Japan, Vietnam, Nigeria and a few other countries to the block list on a couple of sites.
RE: IP Bunch Analysis:
I actually have a report I run similar to your IP Bunch that I call a Proximity Alarm that shows groups of similar class C and D hits that use similar user agents, operate within a similar time frame, or are potential threats due to behavior.
Gotta play hardball with these IP hopping content pirates!