Forum Moderators: phranque

Message Too Old, No Replies

Stopping the Mutant Surge

Can I set a trap for these scraper zombies?

         

BaseVinyl

9:12 pm on Apr 25, 2006 (gmt 0)

10+ Year Member



I type in my unique domain name into a search engine in quotes and get a whole string of sites that are somehow scraping titles and snippets from my website and adding about 50 other scraped titles and snippets from other websites and these little snippets are about two fathoms down from a bunch of advertisements; i won't mention any names but it kinda rhymes with spamsense...so anyway where do these websites grab these snippets and titles from my site?
Do they have drooling robots that run through my site and collect snippets? How do I recognize these beasts and stop them or do they grab these snippets and titles from search engines and stuff...

Is there anything that can be done...some of these sites appear higher up in search results than my own site where they have lifted snippets...gosh...do I need to go a huntin'?

Is there something in my robots text I need to block?

Thanks

2by4

10:43 pm on Apr 25, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



scrapers don't pay attention to robots.txt except as another source of potenially interesting urls.

You have to do a lot more to stop scrapers, bot traps, bot blocking, etc. These forums are filled with various methods.