Welcome to WebmasterWorld Guest from 18.210.28.227

Forum Moderators: martinibuster

Message Too Old, No Replies

Scrapers and Crap directories

     
1:05 am on Feb 26, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 3, 2003
posts:1246
votes: 0


What I'll say is certainly nothing new to anyone here. Scrapers and negligible-value-directories have been capitalizing on adsense since the start. But...I just did a google search for my own keywords and another search on my domain name (instead of clicking "Find web pages that link to" I chose "Find web pages that contain the term") and was stunned at how many of these things (euphemism for parasite) I found.

Almost none of these sites offers any user-value and few have any chance whatsoever of surfacing in serps based on pagerank, anchortext, or any other seo variable. Which leads me to suspect that the majority of clicks being generated by these sites are suspect at the very least.

It astounds me that google continues to let this crap go on. Who suffers? Presently advertisers and publishers (advertisers lowering bids due to smaller ROI). But, at some point, how can this not affect google? This is a branding problem that yahoo/overture are not indulging in themselves.

The solution, to some extent at least, would be to manually review all sites. Makes me wonder if the google posting on craigslist for a quality reviewer had something to do with this.

1:26 am on Feb 26, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 25, 2002
posts:776
votes: 0


few have any chance whatsoever of surfacing in serps

Maybe not in google but go try some searches in Yahoo.

3:16 am on Feb 26, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 1, 2004
posts:79
votes: 0


Who suffers? Presently advertisers and publishers (advertisers lowering bids due to smaller ROI).

Everyone says this but has it ever been proven? If you are in the market to buy some widgets, do a search on google for widgets, end up on a scraper site dedicated to widgets, and click on an adsense ad that says "Click here to buy widgets", wouldn't that have a great chance of converting to a sale?

3:21 am on Feb 26, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Sept 7, 2004
posts:81
votes: 0


Yes, you're right.

Plus these sites wouldn't be making money right? Doesn't smart-pricing reduce profits for anything that doesn't convert decent? While they're not the best sites in the world, they work because people still find exactly what they want.

3:50 am on Feb 26, 2005 (gmt 0)

Senior Member

joined:Oct 27, 2001
posts:10210
votes: 0


Everyone says this but has it ever been proven? If you are in the market to buy some widgets, do a search on google for widgets, end up on a scraper site dedicated to widgets, and click on an adsense ad that says "Click here to buy widgets", wouldn't that have a great chance of converting to a sale?

But what if you aren't in the market to buy widgets? What if you were simply looking for information on widgets? You arrive on a scraper site, and the first five links are in a large, borderless AdSense rectangle that's disguised to match the page. If you're like many users, you'll click the first link or two on the assumption that it's a scraped search result.

Or maybe the content on the page will consist of filler material: "If you're looking for information on keyword, you've come to the right place. This site has information on keyword, forums on keyword, and chat rooms about keyword (etc, etc)." When you encounter fluff text like this, you may click on an ad in the hope of getting the information you're looking for (especially if the owner of the junk page has disabled your browser's back button so you can't return to the legitimate SERP).

Now, some advertisers may feel that they get decent ROI from scraper and other junk sites, and good for them if they do. But other advertisers don't feel that way. And, more to the point, the large untapped market of potential advertisers from Main Street and Madison Avenue are going to be very leery of a medium that appears to be increasingly dominated by pages that wouldn't even be up to the standards of "occupant" mailings or pizza flyers in the offline advertising world.

I suspect that scraper sites and other junk sites are highly visible right now because Google hasn't yet found a way to banish them or to lower their rankings in its SERPs without causing a lot of collateral damage. On the search side, Google probably wants to deal with the problem via algorithms and filters (a "scaleable solution") instead of playing whack-a-mole; on the AdSense side, it may prefer to solve the problem with "smart pricing" (and possibly via automated adjustments to its payout formula), which has the advantage of leaving plenty of "run of network"-quality inventory for price-sensitive advertisers as new and costlier AdSense variants are introduced. Whatever happens, it seems unlikely that Google will let the scrapers and other opportunists take permanent control of its SERPs.

4:51 am on Feb 26, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 1, 2004
posts:79
votes: 0


Now, some advertisers may feel that they get decent ROI from scraper and other junk sites, and good for them if they do. But other advertisers don't feel that way.

Feelings are one thing and facts are another.

I'd like to know if there are any facts to support the negative feelings people have about ROI from scraper sites.

It would be interesting if someone did a study that proves how well scraper sites convert to sales for advertisers.

5:25 am on Feb 26, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 3, 2003
posts:1246
votes: 0


"But what if you aren't in the market to buy widgets? What if you were simply looking for information on widgets?"

This is the case with many info sites, for which the relevant advertisers appearing on the sites seek to interest users by appealing to their relevant needs. In this case, there is no widget, users are not specifically looking for widgets, and, though sales conversions may result via adsense, neither the publisher nor the advertiser is served by scummy sites that attempt to SPAWN the creation of clicks.

"the large untapped market of potential advertisers from Main Street and Madison Avenue are going to be very leery of a medium that appears to be increasingly dominated by pages that wouldn't even be up to the standards of "occupant" mailings or pizza flyers in the offline advertising world."

Yes, this is fundamentally a branding problem. Long-term, google cuts its own throat by allowing this garbage to continue. Which vector would upscale advertisers more likely choose? Adsense with its gazillion rip-off sites or the yahoo/overture network?

"I suspect that scraper sites and other junk sites are highly visible right now because Google hasn't yet found a way to banish them or to lower their rankings in its SERPs without causing a lot of collateral damage"

Algorithmns I don't think will ever cut it. They need more warm bodies---not to hand select entries for their index the way yahoo does/did for it's directory, but rather to weed out the wretched hive of scum and villiany (a la obi wan kenobi) that proliferates under the banner of adsense.

5:34 am on Feb 26, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 3, 2003
posts:1246
votes: 0


"I'd like to know if there are any facts to support the negative feelings people have about ROI from scraper sites."

I don't know how anyone could not look at the legions of bottom dweller sites out there and not conclude that advertisers are being ripped off. Maybe it's just a nibble here and a nibble there, but those nibbles add up. And when an advertiser sees that he keeps getting less return for his advertising budget each month (beeeecause----more scrapers, scummers, and bogus directory clowns kept crawling out from under rocks every stinking minute), what would his reaction be other than to re-evalute the content network?

If you look at roughly 25 or 30 of these sites as I have done this evening (this is just in my own little niche), it is nearly impossible to conclude that these sites 1. benefit users, 2. benefit advertisers, and, in a long-term sense 3. benefit google.

5:57 am on Feb 26, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 1, 2004
posts:79
votes: 0


There is no doubt that junk sites are really annoying for google and for the end user... but like I said, I'd like to see data that shows how well clicks from junk sites turn into sales.
6:31 am on Feb 26, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Nov 5, 2002
posts:413
votes: 0


I would'nt be surprised if if adsense clicks from 'junk' pages actually convert as good or better than pages where the the user is wading though a lot of content.
But... I also believe that many users may start to associate adsense ads with spammy, junky or rip-off pages. That won't be good for advertisers or the adsense program.
6:48 am on Feb 26, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Feb 7, 2003
posts:319
votes: 0


When I am in a user mode and I come across a "scraper" type directory site, I just click the back button instead of even taking the time to peruse the page.

The bit that chafs my hide is when they take the exact text and terms from sites and just paste it below the fold as if it is a directory to assist someone...it is obvious they have not intention of providing any other information than the Adsense ads which is a violation of Googles own TOS. How can that not be construed as a "Made for Adsense" site!

The challenge I see with Google using their algo's is that these "bottom dwellers" have grabbed top keyword phrases from the creme of the crop sites and they run the risk of losing the good with the bad.

My 2 cents.

8:08 am on Feb 26, 2005 (gmt 0)

Senior Member

joined:Oct 27, 2001
posts:10210
votes: 0


I would'nt be surprised if if adsense clicks from 'junk' pages actually convert as good or better than pages where the the user is wading though a lot of content.

Well, some advertisers have said they don't convert at all. Whether the traffic converts or not probably depends on a number of factors. That's why advertisers need controls that go beyond what they have now. Some advertisers are opting out of the content network because the choice is all or nothing--and "all" includes too much junk for their tastes.

As for whether "wading through a lot of content" is good for conversions, that probably depends on what the content is (which is why Google has a sliding scale of "smart pricing" discounts for advertisers). Last April, when smart pricing was introduced, Google used the example of a camera review converting well and a page of photo tips converting poorly. A positive review can "presell" a product, service, destination, hotel, cruise ship, etc. to the reader even if that wasn't the reviewer's objective. (On my own site, for example, some reviews have generated sales for travel vendors that were worth many thousands of dollars per sale, according to PR people and travel agents who have e-mailed me to say "Thanks." That's understandable when you consider that users often consult reviews or articles as part of the decision-making process. Just because they're searching Google for "widgets" doesn't mean they're ready to buy a widget--they may be looking for information to help them make a buying decision or to confirm a tentative decision that they've already made.)

8:13 am on Feb 26, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 6, 2004
posts:503
votes: 0


I recently had someone ask me for a link exchange with their site. When I went to visit their site, I found a pretty crappy "Adsense scraper" site.

Needless to say, I did not give them a link.

9:25 am on Feb 26, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 21, 2005
posts:2264
votes: 0


I suspect that scraper sites and other junk sites are highly visible right now because Google hasn't yet found a way to banish them or to lower their rankings in its SERPs without causing a lot of collateral damage.

...or Google has the technology but has an undisclosed reason why they'd like scrapers to continue.

12:31 pm on Feb 26, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 28, 2004
posts:105
votes: 0


Google takes care of it by smart pricing. No, I agree,not the optimal solution, but a good one.

On the whole,if an ad converts it fulfills it purpose no matter where is presented from, if it doesn't, it gets an increasing lower value.

As far as adsense reputation being diluted by useless directories etc, well that's more of a complex debatable issue. I am sure google has taken it into account.

12:55 pm on Feb 26, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 21, 2005
posts:2264
votes: 0


Google takes care of it by smart pricing

...would have been a valid argument if Google the Adsense company was different to the company responsible for the SERPs.

1:42 pm on Feb 26, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 22, 2004
posts:73
votes: 0


The scrapers will be purged. Shareholders will demand this. Scrapers damage the brand that is Google.

It's just a matter of time. A concerted effort could have this cleaned up, right quick.

As far as an algo approach, all it takes is to find ONE site. A scraper will rarely have just one. Find that one and it will lead to many others, all connected by the AdSense publisher account.

Flip that one switch and many bad lights will go out.

4:30 pm on Feb 26, 2005 (gmt 0)

Preferred Member from CH 

10+ Year Member

joined:Mar 10, 2004
posts:429
votes: 0


Adsense kind of reminds me of the pay for play scheme over at mp3.com a few years back, but on a massive scale.

Back in the late 90's musicians were awarded a slice of a million dollar pie every month, depending on how many downloads, streams they received.

Once scammers found they could make money from mp3.com downloads you had DJ squeaky from dusseldorf, uploading his mp3 masterpieces composed lovingly (in 3 minutes) on his 15 euro casio keyboard.

Then he'd either get tons of other friends, family or artists to play the songs, or point a bot at the page and milk monthly checks. Some indies were making 25 grand a month (and more) from schemes like that on mp3.com.

Suddenly regular surfers checking the top of the mp3.com charts would find DJ squeaky reigning supreme for 3 months at #1 - They thought the site was a joke and left...Then you just had the artists.

That is until the banner advertisers realised that most of the hundreds of thousands of page views on mp3.com were just artists streaming each others songs for pay for play.

They ended up slashing ad budgets on mp3.com, who then started looking at other ways to monetize and ended up allowing users to upload copyrighted songs and play them from their mymp3 lockers.....The end.

I see a comparison with Adsense.

Today publishers are fighting for a billion dollar pie every month, depending on how many Click throughs they receive. Now you have Webmaster Squeaky from Dussledorf, trading in his 15 euro casio for a content cranker.

Once advertisers realise their ads are being shown on scraper sites, with 3 blocks of ads above the fold many of them may pull out of the program....Lessening revenue for publishers offering original content, and lowering the integrity of the entire program.

Allowing scraper sites is short term gain for long term pain (just my opinion)

5:24 pm on Feb 26, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 3, 2003
posts:1246
votes: 0


"Allowing scraper sites is short term gain for long term pain (just my opinion)"

"Scrapers damage the brand that is Google."

Just my opinion, but I'd agree with this completely. The feeling I get when I arrive at a page that has 3 blocks of adsense and only scraped content (and even if the content is not scraped it still isn't helpful genuine, or trustworthy) is very similar to the feeling I had about altavista when that SE started to slide downhill. I didn't feel as though I could trust the results.

Google has a lot of goodwill going for it. But searchers tend to get fed up with blatant attempts to direct them to ads instead of the content they were actually looking for (the *topzilla TV ad is very effective because people HATE popups). When adsense works "right", the ads are unobtrusive and users and advertisers become joined in a click action in a fairly natural way. But when the sole purpose of a page is to display an ad and manipulate user behavior to generate a click...
(I know there's a lot of arugment as to what a made for adsense site is, but I think the way to resolve that issue is to simply ask the question: "would the site exist if adsense was not on the page i.e. does the site have intrinsic value to users?")

It doesn't matter that a percentage of clicks on scum sites will result in conversion. The long-term effect for google will be damaging i.e. guilt by association. Because every time a user looks for information and comes across a bogus site that bears the title "Ads by Google", it just tarnishes the image of the search engine and the ad program further. It is a cumulative effect that will accelerate as more of these sites come online (which they do every few minutes I suppose).

For me, it's hard to believe google would be stupid enough not to see this. If I were Yahoo, I would launch a well-staffed effort to rid yahoo of adsense-scum-sites simply to differentiate the quality of the yahoo results versus the quality of the google results (i prefer google, but if I were yahoo, that might make good marketing sense).

6:50 pm on Feb 26, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 25, 2002
posts:776
votes: 0


Google takes care of it by smart pricing

I'm continually amazed by how much blind faith people put into "smart pricing." How can anyone (including people at Google) know how well it works. Outside of Google there is more speculation about how it works than disclosures. For all we know the scraper sites get a smart pricing boost.

6:57 pm on Feb 26, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Mar 5, 2004
posts:215
votes: 0


If they get a smart pricing boost, then they are converting well. So why replace them with sites which doesn't convert as well?
1:16 am on Feb 27, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member annej is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Feb 17, 2002
posts:3318
votes: 0


[quote]users may start to associate adsense ads with spammy, junky or rip-off pages[quote]

My concern exactly!

If Google is going to continue to support scraper sites I wish they would start a new program for those of us who have content sites. Something with a different look and name.

The then trash ads could put their ads on scraper sites and all of us with true content or selling real products could be on a different system.

1:45 am on Feb 27, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 19, 2000
posts:2501
votes: 27


As an avid Google fan, I am truly disappointed to say that I have started to use other engines when searching for personal reasons. These sites are more than very annoying ... they are worthless to me as a searcher and cause nothing but frustration. They often occupy the majority of the top ten results. I've had it!

When Google gets rid of these things ... I'll come back. Until then, I want real "content" and will find it elsewhere.

Signed: Ticked off in Tortola (where most shopping is done on the internet!)

3:08 am on Feb 27, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 1, 2004
posts:79
votes: 0


Is it just me or are there more of these sites in Yahoo?
3:57 am on Feb 27, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 4, 2002
posts:1687
votes: 0


As an avid Google fan, I am truly disappointed to say that I have started to use other engines when searching for personal reasons.

I've been doing the same myself lately. Enough ridiculous serps from G, when searching for my own purposes, made me realize that it isn't necessarily the best SE anymore. I've got address bar shortcuts for Y and MSN in Firefox, and I've been using them a lot the last while. Sometimes it's just the same dross, but sometimes it's better. G has lost its edge though, no doubt.

6:59 am on Feb 27, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member jetteroheller is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 22, 2005
posts:3062
votes: 6


[ As an avid Google fan, I am truly disappointed to say that I have started to use other engines when searching for personal reasons. These sites are more than very annoying ... they are worthless to me as a searcher and cause nothing but frustration. They often occupy the majority of the top ten results. I've had it! ]

March 2004, Google destroied my real estate web sites business by a change.

I made a test with
"name of an importang city, real property"

I visited search result 1..5
I evaluated each search result for
4 points extrem usefull
0 pure spam

The points have been multiplied by
10,6,4,3,2 for place 1..5

All the other search engines made 41 to 62 of 100 possible points.
Most of them no spam result in the first 5,
or only 1 spam result in the first 5

Only Google was the big exception
4 spam results and one big list of
real estate companies, no real property. 2 points only

When I repeat today the test with Google

1.) Spam result with Google AdSense
2.) 3 Points
3.) 3 Points
4.) 4 Points
5.) 1 Point

So 44 Points from 100

10:49 am on Feb 27, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 11, 2004
posts:1062
votes: 0


It would be interesting if someone did a study that proves how well scraper sites convert to sales for advertisers.

Even if they did do a study that proved to scraper webmasters that their sites converted poorly, they would still find some sort of bass ackwards justification for polluting the internet with their garbage.

The scrapers will be purged. Shareholders will demand this.

I am not sure the shareholders will demand this, but advertisers and non-scraper webmasters are putting the pressure on Google to make a change. Hardly a day goes by on this forum or any other where there are not loud voices protesting scraper "search engines and directories."

Not long ago, it seems, Google wouldn't allow AdSense on "search result" pages. Now they do. Hopefully, with pressure from advertisers and webmasters, they will rescind this change.

If that doesn't work, Adword Advertisers will use the democratic approach of blocking these sites when this option is made available in the future.

Google has always tried to deal with spam and junk sites with an automated approach and have by and large, failed most of the time. If that is their sole solution to scraper sites, then they will fall short of their goal.

Financially speaking it makes more sense for Google to initiate stronger quality control for Publishers opening the way for Adwords advertisers to turn the content option back on. They are trading the small profits that scraper and junk (automated content generated) sites bring in, and losing out on having more Adword advertisers come back to AdSense (which would be more protitable).

And let's not forget the damage they do to their brand with their fast and loose publisher quality control.

Although slow to act on advertiser and publisher wish list requests, Google does listen. It just takes them a long time, it seems, to get on board.

Threads like this one, as well as lucid and well reasoned posts and emails to Google, will be required on a frequent basis, albeit ad nauseum, to keep the pressure on Google.

Jenstar and senior members will probably get sick of the topic, but it must be kept in the forefront in order to keep the pressure on Google to make a change.

IMO

12:06 pm on Feb 27, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 21, 2005
posts:2264
votes: 0


Freedom, I hear what what you are saying but it would be wise to work on the basis that Google is not going to do anything about scrapers (in the short term at least). If they had a commitment to that they would have done something already.

As a publicly limited company they are accountable to shareholders. If a certain percentage of scrapers in the SERPS (say 10%) does not affect their marketshare and makes them more money (it's rare for an advertiser to pull out on a matter of principle that there are scrapers in the SERPs) then Google will continue to "devote" 10% of their SERPs to scrapers. The point worth discussing here is what we can do to adapt how we work considering that scrapers are a "given".

12:38 pm on Feb 27, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Mar 5, 2004
posts:215
votes: 0


@freedom.

I guess that google didn't have enought processing power in the beginning to handle all the dynamic generated search result pages (most of them are visited very rarely more than once so it wasn't worth to index them just for one page view.), so they just wanted to skip them in the beginning. I think that was the reason.

1:33 pm on Feb 27, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Jan 21, 2005
posts:298
votes: 2


The problem is that we are looking at those scrapper websites from a webmaster's point of view. For an average Internet user those sites come across as simple directories. This for Google is just "good enough" to keep because even if one out of ten clicks convert it's money for them and smart pricing takes care of the rest.

Frankly I don't think much about those scrapper sites. They are not something we should worry about... Why?

Well... for one it'll do us no good.. other than keeping us raged about those websites. They are really nothing to worry about... as long as you have a good website focused on what you like the best you will be in a decent position to reap the rewards... from adsense.. or from other affiliate deals.

But yes, I do find it pathetic on Google's part that those scrapper sites rank on the first page for many-many queries.

Three possible conclusions:
1) Maybe those sites are converting better for google and they like the money.

2)Google is doing a sorting out all the scrapper sites.. they find the scrapper sites on their index. So for some time they rank well and then turn it around to remove all of them.

3) Those scrappers drive traffic using adwords. Thus more money for Google.

Just a few thoughts...

This 112 message thread spans 4 pages: 112