Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: not2easy
The problem is that almost daily I find other sites (usually affiliate sites) that have copied my text. Now I have a new part-time job: contacting scrapers to tell them to remove my content. It is very time consuming and I am sure it has hurt our SERPS. Any suggestions?
If it's one loser trying to scrape your site to post on his own site, then, yeah, an IP block (at least a temporary one) will probably be sufficient.
But if you're talking about somebody who is deep into malicious behavior, then, no, probably not. You'd need to have other protections in force (such as banning certain known scraper agents).
To a certain extent, if you've posted something online, people can steal it. We just do the best we can to try to keep ahead of it.
Would banning the ips really work? Don't these bad guys do things to spoof their ip addresses?
That's why I said it's a never-ending task. :)
Sure, they do all sorts of stuff to circumvent any security measures you put in place, or they'll use another IP and try again. But that's no reason not to at least try and keep them out.
Start by reading Forum 11 [webmasterworld.com] which will point you to some of the bad guys, bad bots and nuisances. I've learned a lot from reading this forum.
Then go through your logs. You'll find some more, often pretending to be visitors. As an example, maybe you'll notice a lot of IPs from a country which oughtn't to be interested in your content. Is your site on Elbonian Widgets really of such interest to China or Korea? Chances are they're scraping your site. Better sometimes to ban a few innocent IPs than get scraped.
You can easily use free DNS-based block lists (eg SPAMHAUS and SORBS) to block open proxies and compromised machines/bots by IP, and you can use the NOARCHIVE robots tag to avoid your content being cached by the SEs (from where it can also be stolen).
If you maintain a small manual IP block list on top then you will make scraping much harder, and the culprits easier to identify.