---- Why does Google AdSense sponsor "scraper" spam sites
scraperrob - 5:42 pm on Mar 29, 2005 (gmt 0)
There is one very important difference between scraper sites and search engines; it's likely sometime in the past you submitted your site to the major search engines. Remember doing that?
You don't have to submit your sites to search engines for them to do their scraping. The Robots.Txt standard says that if you are not dissallowed then you can scrape all you want. Like I said, ban all robots and your site will be safe from scraper sites like mine. I will still outrank you if you are in my niche.. but at least I won't advertise your url.