Big changes promised shortly at Google

[Sgt_Kickaxe]

Moultano, a purported Google employee, posted the following a few hours ago in response to questions about adsense spammers getting removed from the program, the decision process etc... [news.ycombinator.com...]

Question:

Even so, it may have affected resource decisions. At the organizational level, did the specter of losing tens of millions in ad revenue had something to do with why Google waited so long to start to address the problem?

Moultano answer:

At the organizational level, Google is essentially chaos. In search quality in particular, once you've demonstrated that you can do useful stuff on your own, you're pretty much free to work on whatever you think is important. I don't think there's even a mechanism for shifting priorities like that.

We've been working on this issue for a long time, and made some progress. These efforts started long before the recent spat of news articles. I've personally been working on it for over a year. The central issue is that it's very difficult to make changes that sacrifice "on-topic-ness" for "good-ness" that don't make the results in general worse. You can expect some big changes here very shortly though.

I'd be interested to hear your ideas on what "big changes" might entail. Earlier in the week Moultano also suggested that Google plans to "dramatically" change how they use links in the near future. If he's right on both counts is the website rankings landscape in for another shakedown at Google?

[tedster]

I once put Chitika on a page - and took it off within 24 hours. They were serving really bad spammy ads about weight loss, that had nothing to do with my content which was technical, and made the page look really rotton. I'm sure lots of other people have had the same experience.

You're right, that is definitely the current situation. I'm saying the quality and nature of the alternative online ad market would change very quickly if Google used Adsense as a negative organic signal in some way.

I don't see Google moving too far from what they do now - if a page is dropped from the organic index as spam, it also gets dropped from the Adsense program. I can't see Adsense being part of the new heuristic that moultano says they've developed.

I'm thinking more along the lines of user data with the possibility of some natural language processing thrown in. And of course, if content farms do what Demand Media says they are doing - actually upgrade the content - then the game changes for the better. I really don't care if the page was created by a contract writer as long as it gives me accurate information that is relevant to my query's intention... well, maybe I do sometimes.

I care about finding authority websites in legal, medical, scientific fields and the like. I care about finding the original source of a news story. Maybe moultano and friends have found a way to improve trust and/or authority.

[jimbeetle]

If Google used anything about Adsense as a negative organic signal...

Of course, that's where G's stuck between the rock and the proverbial hard place. Use our product to make money, but...

And going back, both the AdSense and Blogger initiatives were championed -- especially by Sergey -- as a way to encourage content creation. Googlebot was hungry and both programs were seen as a way toward the "organizing" part of G's original mission.

Would be a bit of a slap in a Founder's face.

[tedster]

a way to encourage content creation

Last year Google even published a list of content gaps they'd identified. I wondered about that at the time - seemed like an open invitation to MFAs and Content Farms.

[AlyssaS]

I'm saying the quality and nature of the alternative online ad market would change very quickly if Google used Adsense as a negative organic signal in some way.

Only if the advertisers complained. It doesn't matter if the publisher is upset by being downgraded in the SERPS, the success of an ad network and the migration of advertisers from one network to another, depends on advertisers.

I'm not sure Google advertisers would be upset if some of the bad content farms were zapped. Look at it from their point of view - they are paying good money for a lead, it's a big part of their budget - but may be getting clicks simply because the ad layout is such that clicking by accident happens a lot, or a lot of people clicking are looking for info, rather than clicking to buy i.e. the page didn't do a great job of filtering. That was the whole reason G cleaned out some of the worst Adsense publishers - it was because they were upsetting the advertisers. Zapping the worst content farms may increase the return on advertising for the advertiser.

If the advertisers remain happy with G and Adwords/Adsense, then even though publishers might sign up with other networks, it won't do them the slightest good if those networks don't pull in the advertisers too. They'll be served up with irrelevant ads (from lack of advertisers) and won't make a bean.

G has the ability to upset publishers and webmasters at will as long as their advertisers stay loyal and the searching public stays loyal.

[Planet13]

Last year Google even published a list of content gaps they'd identified.

Would you have a link to that post?

[Spencer]

If Google were really worried so much about the quality of sites, why not just add an adsense sidebar to every website linked to from Google and do away with the intermediates ?

[tedster]

Would you have a link to that post?

My apology - it looks like my memory was a bit off and they didn't actually publish a list. Google applied for a patent last year on the process (as did Demand Media.) See Google patent for identifying inadequate content areas on the web [webmasterworld.com].

[Simsi]

Just a side-step. The quote from the OP by Moultano doesn't directly mention Adsense, even though it's responding to a question on advertising. Also, wasn't there another quote that said Google are looking at the way linking contributes to rankings.

My point being, is this not what the "big changes" are possibly relating to and isn't the whole Adsense thing just a (potential) side-show to the (potential) main event here?

[AlyssaS]

My point being, is this not what the "big changes" are possibly relating to and isn't the whole Adsense thing just a (potential) side-show to the (potential) main event here?

You are right - we all got well and truly sidetracked by Brett Tabke's mischievous comment about filtering by Adsense.

Back on topic, am noticing something odd today. I visit a blog of a friend regularly, and last August, he bought a new domain with his name in it (his old domain just had one of those "fun" names everyone used to get seven years ago). Being too lazy to do a redirect he just put a link on the latest post of the old to the new, and then posted on the new.

Within weeks of the new domain going live, G started to list the new domain at #1 for his name - presumably because lots of people changed their blogroll links.

But today, suddenly there's the old domain at the top of the SERPs when you do a search for his name, which I automatically clicked and thought hello, I'm on the wrong site.

I don't know why they'd suddenly list the old domain - it doesn't have his name in the domain or title, but it is over seven years old, and probably has some long standing links. Note that it's also inconceivable that anyone is building new links to his old site - it's a personal site, and everyone knows he has a new blog.

So, trying to extrapolate from my one lonely anecdote (which I know I shouldn't do!) - could part of the big changes be a move away from the over-emphasis on new, and giving more weight to old links and old sites?

I would imagine that spam links have a short life-span. The blogs or forums they are created on die. So if you did a filter and weighted for stuff that is say, over ten months old, then you could filter out all the noise. I picked ten months because I read somewhere that G did a study that shows many sites do not last more than ten months.

[Reno]

and giving more weight to old links and old sites...
many sites do not last more than ten months

I hope you are right about the first part of that quote ~ it only makes sense that older sites in many (if not most) cases will have a degree of authority simply by virtue of their continuation. The logic being that if a site is still available, with reasonable updating, after 4 or 5 years, then it must have something going for it.

Re the 10 month expiration, that seems too short IMO. People buy domains for a year, and most hosting is annual, so it seems reasonable that a more accurate figure would be in the 24 month range. The thought being, they go online, give it a year, it's not working so they try one more year, it's still not working so they pull the plug. I have not much evidence for that, but I will say this: One of my sites is an arts directory that had over 3000 listings. I spent the last 5 days checking for deadlinks and found 500. I hadn't had the opportunity to do a thorough check for some time due to other issues, but even so, it was clear to me that a bunch of people are giving up on the internet as an income stream.

........................

[AlyssaS]

Re the 10 month expiration, that seems too short IMO. People buy domains for a year, and most hosting is annual, so it seems reasonable that a more accurate figure would be in the 24 month range. The thought being, they go online, give it a year, it's not working so they try one more year, it's still not working so they pull the plug.

Perhaps.

My reasoning was this: spammers look for dead blogs and forums to post their links on (sites that are maintained simply zap the spam). So, they are dropping links on sites that are perhaps due to expire - people don't bother to renew domains that they arn't maintaining. So - the links will always have been created in the ten months or so before expiry. As the old set of links drops off, because the domains arn't renewed, they are creating fresh links on domains still with a bit of time to expiry, so they sort of maintain their position in the SERPs because the total number of links they have stays the same.

But if you placed a filter on the age of the links, they would end up with zero, as all these short term links would not be counted.

Am I being too simplistic?

[Elsmarc]

@MrFewkes

If I worked for google - I would clean it up thats for certain.

As a programmer - I KNOW how easy it is.

I would say that if you could actually do it so easily, Google would hire you in a New York Second at one heck of a 6 figure salary.

[Elsmarc]

and giving more weight to old links and old sites...
many sites do not last more than ten months

Actually the issue is how far into the future is the domain paid for rather than how old it is. I pay my domains out every 5 years for another 5 or more. All of my domains have expiration dates of 2019 right now. That shows I'm in this for the long term. In 2015 I will add another 5 years. Maybe 10 years. It's cheap and - As I said - I believe it is a signal that the person / company is *serious* about a long term commitment.

[rowtc2]

I think the fight with content farms must be related to scrapper sites, example a news site which does not have original content or added value and only publish thousands of feeds.

Regarding sites like Demand Media which are often on the serps, i think things must be related to anti-monopoly policy. There are a lot of simple sites whithout advanced SEO on it and with useful content.

I hope they will keep the serps free, various sources of informations and no monopoly.

[Planet13]

All of my domains have expiration dates of 2019 right now. That shows I'm in this for the long term.

FWIW: Matt Cutts has said that google DOESN'T use that as a factor in ranking.

There is a video on youtube where he specifically addresses this issue.

[koan]

Actually the issue is how far into the future is the domain paid for rather

You can throw money on this to make people think the site is serious, you can't throw money on real history. I've had sites for 10 years and I just renew every year for 1 year, for me it's like my cable TV bill.. don't see the point of paying years in advance.

As a programmer - I KNOW how easy it is.

No offense but your confidence seems to be based on the lack of awareness of the complexity of the issue. Might be easy considering just your own interests, but Google has to think of thousand of other situations in play.

[tedster]

Matt Cutts has said that google DOESN'T use that as a factor in ranking.

He spoke about that recently, too. Here's the link to the video: [youtube.com...]

The idea that length of domain registration is a factor got rolling after Google mentioned it in their Historical Data Patent [webmasterworld.com] and it might keep rolling for a few more years, just like other SEO mythology tends to. I'd prefer to kill it.

[Simsi]

You can see Google's problem here. They want the best SERPS... or at least they want the best balance of SERPS and earnings... yet they are aware that because links count in the algo, SERPS can and are manipulated.

Google's biggest issue is probably cultural. Niches like pron, gambling, pills and maybe even dating are lucrative largely because social and political velues drive them below the waterline and your average joe schmo probably won't want to naturally link to sites in these niches because they want to keep their social habits private. Hence I would imagine the vast majority of links in these niches are exchanged, paid for or in some way obtained "unnaturally".

Resultingly, it not only makes sense that Google treats different verticals in different ways but that these niches are where most of the spam problems are going to crop up. The problem is that regardless of the social implications, some of these sites are genuiniely useful to a large groups of people.

I would have thought that any changes to linking would (primarily) approach these sectors and aim at trying to identify where linking is unnatural. How, I have no idea how - that's the challenge - but I wouldn't be surprised to see any changes to the algos affect the impact of links, in these niches particularly.

[AlyssaS]

Hence I would imagine the vast majority of links in these niches are exchanged, paid for or in some way obtained "unnaturally".

They use two evil tools called Scrapebox and Xrumer to create their links (google them to find out the true horror).

Scrapebox in particular really uses up a ton of bandwidth - you can tell when they are trying to hammer your site because page loading slows way down. I've taken to banning IP addresses to keep them away - they use proxies, so I just ban the IP of the proxy so no one can use it again (unfortunately there are loads of these proxies, so it's a never ending battle).

Once they've scraped all your urls, they load them into Xrumer which then starts to post automatic comments.

On maintained blogs, these go straight into the spam trap - On Wordpress, Akismet is getting better and better at stopping them.

But like the email spammers, because they are doing this by bot, they only need say 2% of their comments to "stick". They usually stick on a blog someone has abandoned, or one where the owner has neglected to update their spam-traps and only logs in once in a while to zap stuff.

Because G gives credit to links immediately, by the time the links are zapped or have dropped off due to the sites they are on expiring, they have created 1000's more.

If G could just put a moratorium on the time new links are acknowledged - say four months - then a) these guys would have to wait to see a return on their activities and b) by the time the moratorium was up some of the links would have been zapped by the site owners or otherwise dropped off.

[Broadway]

I would think that the type of MFAs that I personally find so offensive would have:

A high bounce rate.
More Adsense showing above the fold than text.
Duplicate / scraped content.
No links from authority sites.

Just approaching a clean up using some type of mix of these factors would produce a big improvement in the SERPs. DemandMedia pages might arguably have some merit but gosh, can't Google even get rid of the sites that 99% of us would agree have no value?


  

[Broadway]

Adsense already Smart Prices us based on click quality. And clearly Google has some idea of what it considers to be Search Quality.

Possibly further penalize (Smart Price) publishers due to a low Search Quality score. Publishers could still crank out the garbage, it's just that the Adense clicks their pages generate have far less value.

[ChanandlerBong]

I have a feeling Smart Pricing II is about to hit the fan, and as we all know, the collateral damage will be huge. I look forward to the resulting 25-page thread on here.

[Edge]

For me the real challenge that I think Google has is that the AdSense and search algo folks are not on the same page.

By most measures I’m aware of Google has created this spam-athon environment by giving AdSense accounts to – well anybody. If one is an ambitious content spammer and creates 200 spammy-content-scraping-whatever sites, and gets random traffic for $10 a month off each site = $2000.

The spam motive is money and the AdSense folks are in many cases the enablers.

[crobb305]

I think this quote is spot on:
"In search quality in particular, once you've demonstrated that you can do useful stuff on your own, you're pretty much free to work on whatever you think is important."

There are so many different indexes all over the world, probably the result of individual employees "doing their own thing", the entire search is a mess, both in terms of quality and consistency.

I'll also add that unnecessary "improvements" like Google Instant, trying to guess or suggest search phrases throws people off track from their original search. I can't tell you how many times I have started to search for something and ended up reading about completely unrelated material thanks to the silly suggestions. Yeah, that's my fault, because I get sidetracked and lose my train of thought easily, but it seems Google's revenue could suffer because of this. Obviously, I can't prove that, but I can tell from my site stats that since it was implemented, my traffic is down and my bounce rate is up. Clearly, the traffic coming to me from Google is less targeted and very poor quality.

Just a thought.

[freejung]

All generalizations are false.

I'm not going to speak for myself, I don't presume to judge the quailty of my own content.

However, there's a guy in my niche who is absolutely undeniably top-notch. He's won numerous prestigious awards, had his content featured in top-tier magazines, had documentary films made about him, the works. His content is fantastic, and his SEO is good too, he ranks for a wide variety of high-traffic keywords and gets vast amounts of traffic.

He runs adsense smack on the heat map. I bet he makes a lot more from adsense than he does by selling his work, and his work is excellent and quite expensive.

Google sometimes throws out a few babies with the bathwater, but the sort of simple-minded fixes some are suggesting here would throw out a whole generation of babies and in the long run do very little to reduce the quantity of bathwater.

"Think before you say these things, Mitch."

[Planet13]

I'll also add that unnecessary "improvements" like Google Instant, trying to guess or suggest search phrases throws people off track from their original search.

Maybe that is the next target of spammers; posting millions of fake queries to google so that their desired keywords appear as the first suggestion in the auto completion list.

[Sgt_Kickaxe]

Lots of possibilities have been mentioned so far. One not mentioned, yet, could Moultano have been hinting at webmaster profiling? Some webmasters place adsense on quality sites, others make MFA sites. Websites, much like cars, are produced by various "manufacturers" and while some of those are known for producing top quality products others are known to cut corners (in one country or another).

Profiling webmasters might be a first step method to get an idea on the odds a site will be of quality or not. Give the benefit of the doubt to a manufacturer with a good rep, much like the credit industry does?

The question would then become "how is google measuring webmaster quality?".

Other ideas ?

[MrFewkes]

Its a total doddle to sort out the junk - theres no argument at all programming wise. Its easy - data volumes mean nothing - permutations do - and there are not that many - maybe a couple of hundred thousand tops.

I am not needed at google to do this - (as one person said they would give me a 6 figure sum :) )

No - google cannot get rid of the problem because of the revenue - thats nothing to do with expertise. They HAVE to keep the bathwater and the baby - because they need the money from the bathwater - irrespective of whether or not they get more from adwords.

I am afraid - that I may have been misunderstood - my solution should I be working at google - would cost google money - but would improve what users get to see.

[edited by: MrFewkes at 7:13 pm (utc) on Feb 7, 2011]

[Sgt_Kickaxe]

Mr Fewkes

No - google cannot get rid of the problem because of the revenue

.

That might be true with some companies but I think Google has shown that they will, whenever possible, take the higher road and make sacrifices to obtain a better product. True, it is a Wall st company, but you have to give them at least that much credit given their commitment to quality improvements. They have the funds to make a better product happen so...

I'm hoping to read more ideas, given the current lay of the land, on what "changes" might reasonably entail (Moultano hinted that they may expand beyond adsense, to links etc) and on what we should brace for. None of us know if our current link building efforts are a waste of time or not, assuming "big changes" are really around the corner.

[MrFewkes]

Yes - it is the links thing that is the most worrying.

I will list what I ***think*** are link related - any of these could change.

Anchor Text
Source page Title and content match
Link Age
Source page trust
Deep link : index page ratio at target
Source page link is deep gain more weight
Target page is deep gain more weight (im doubtful of this though)
Image links gain weight lose weight?
Identical anchor % tolerance?
Identical anchor contained in mixed string tolerance?
Part match anchors
TLD power changes
Article/Blog/News/Video changes strength
Network level anchors (including the above) - so site a) to site b) to site c) with the same anchor variations giving c) boost due to a to b pathway - may even be deeper.

I dont think this change though - when it comes - will be as big as say the Florida update in terms of disruption of serps.