grandma_genie - 11:13 pm on Apr 10, 2011 (gmt 0)
I know this topic has been discussed here on Webmasterworld for years. I have read the posts and am even more discouraged than ever. My site is hosted, so I am limited in what I can do. I use htaccess to block by IP, user agent and query string. But the attacks are coming in greater numbers. Today the content of my whole site was stolen. The scraper came in from one referer, IP address kept changing as well as user agent. There was nothing other than the referer that could be blocked, but the various IPs went from one directory to the next, downloading everything including images and text. I am assuming this was an attack from a Zombie botnet. These content scrapers probably have millions of zombie computers to choose from. They set up their software to rotate from one to the next, stealing as they go. The IPs were not from server farms. I know about speed traps and volume traps, but I don't think that would work in this case, because each IP came and just took one section, then the next IP came in and took another one. What is a webmaster to do who can't write scripting languages and is feeling overwhelmed?