Scrapers Copying Text Daily

Copyright Infringement

[Steph_R]

To better our search engine rankings, we hired two new writers who are busy re-writing the text on our site. They are doing a really nice job, and it is very expensive.

The problem is that almost daily I find other sites (usually affiliate sites) that have copied my text. Now I have a new part-time job: contacting scrapers to tell them to remove my content. It is very time consuming and I am sure it has hurt our SERPS. Any suggestions?

[freewebsiteideas]

I share your pain and I would love to know a solution to this problem as well.

[stapel]

I wouldn't bother contacting the scrapers. Why ask the "bad guys" please to be nice?

Instead, contact their web hosts, and get the page, maybe even the site, taken down.

In my experience, doing the latter is generally much quicker.

Eliz.

[malachite]

As well as the advice offered by stapel, you could also keep an eye on your logs and start banning the IPs of the bad guys so they can't scrape any more of your content.

Caveat: This is a never-ending, but worthwhile job ;-)

[freewebsiteideas]

Would banning the ips really work? Don't these bad guys do things to spoof their ip addresses?

[stapel]

Depends on who the "these guys" are.

If it's one loser trying to scrape your site to post on his own site, then, yeah, an IP block (at least a temporary one) will probably be sufficient.

But if you're talking about somebody who is deep into malicious behavior, then, no, probably not. You'd need to have other protections in force (such as banning certain known scraper agents).

To a certain extent, if you've posted something online, people can steal it. We just do the best we can to try to keep ahead of it.

Eliz.

[malachite]

Would banning the ips really work? Don't these bad guys do things to spoof their ip addresses?

That's why I said it's a never-ending task. :)

Sure, they do all sorts of stuff to circumvent any security measures you put in place, or they'll use another IP and try again. But that's no reason not to at least try and keep them out.

Start by reading Forum 11 [webmasterworld.com] which will point you to some of the bad guys, bad bots and nuisances. I've learned a lot from reading this forum.

Then go through your logs. You'll find some more, often pretending to be visitors. As an example, maybe you'll notice a lot of IPs from a country which oughtn't to be interested in your content. Is your site on Elbonian Widgets really of such interest to China or Korea? Chances are they're scraping your site. Better sometimes to ban a few innocent IPs than get scraped.

[DamonHD]

Hi,

You can easily use free DNS-based block lists (eg SPAMHAUS and SORBS) to block open proxies and compromised machines/bots by IP, and you can use the NOARCHIVE robots tag to avoid your content being cached by the SEs (from where it can also be stolen).

If you maintain a small manual IP block list on top then you will make scraping much harder, and the culprits easier to identify.

Rgds

Damon

[wmuser]

Contact their hosts,not themselves