Matt Cutts and Amit Singhal Share Insider Detail on Panda Update

[tedster]

Senior member g1smd pointed out this link in another thread - and it's a juicy one. The Panda That Hates Farms [wired.com]

Wired Magazine interviewed both Matt Cutts and Amit Singhal and in the process got some helpful insight into the Farm Update. I note that some of the speculation we've had at WebmasterWorld is confirmed:

Outside quality raters were involved at the beginning

...we used our standard evaluation system that we've developed, where we basically sent out documents to outside testers. Then we asked the raters questions like: "Would you be comfortable giving this site your credit card? Would you be comfortable giving medicine prescribed by this site to your kids?"

Excessive ads were part of the early definition

There was an engineer who came up with a rigorous set of questions, everything from. "Do you consider this site to be authoritative? Would it be okay if this was in a magazine? Does this site have excessive ads?"

The update is algorithmic, not manual

...we actually came up with a classifier to say, okay, IRS or Wikipedia or New York Times is over on this side, and the low-quality sites are over on this side. And you can really see mathematical reasons.

[tedster]

An interesting point about humans checking on the algorithm:

you look for signals that recreate that same intuition, that same experience that you have as an engineer and that users have. Whenever we look at the most blocked sites, it did match our intuition and experience, but the key is, you also have your experience of the sorts of sites that are going to be adding value for users versus not adding value for users.

[gotmetoo]

Does this site have excessive ads?

Hm... interesting, given rather strict guidelines on the number of ads anyone faces while employing AdSense, for instance, on their site, this statement sounds odd coming from google.

[incrediBILL]

Don't forget that the search team and AdSense team are at odds with each other most of the time, probably diametrically opposed in philosophy of what constitutes a "good site".

The search team's main goal is to keep the searcher at Google so they get more face time with AdWords, they could give a tinker about AdSense if they can keep us off our sites and on theirs.

Another reason I advocate everyone use NOARCHIVE as cached pages gives Google more face time and less to your site.

[chrisv1963]

Don't forget that the search team and AdSense team are at odds with each other most of the time, probably diametrically opposed in philosophy of what constitutes a "good site".

Correct. I received an (automated?) email from Adsense today because they noticed that I'm running less than three ad units on my pages. They recommend to add more ad units. It sounds like a bad joke after I lost 40% of my US traffic, possibly because of too many ads in the eyes of Google's search team.

[Dead_Elvis]

Great find! I actually found the article heartening. This quote in particular struck me...

our classifier that we built this time does a very good job of finding low-quality sites. It was more cautious with mixed-quality sites, because caution is important.

My site took about a 30 to 40% hit. Some sites took massive hits of 80 or 90%. I'm hoping this means my site was seen as "mixed-quality" and not "poor quality."

Gotta find a silver-lining somewhere. lol

[tristanperry]

It also does seem like whilst there might be some tweaks here or there, the big changes are here to stay.

Wired.com: Do you feel that this update has done what you wanted it to do?

Cutts: I would say so. [...]

Singhal: It’s really doing what we said it would do.

Cutts: Which isn’t to say we won’t look at feedback.

And also further proof/evidence that this is an algorithmic change and not a penalty:

Cutts: If someone has a specific question about, for example, why a site dropped, I think it’s fair and justifiable and defensible to tell them why that site dropped. But for example, our most recent algorithm does contain signals that can be gamed. If that one were 100 percent transparent, the bad guys would know how to optimize their way back into the rankings.

-----------------------------------

Correct. I received an (automated?) email from Adsense today because they noticed that I'm running less than three ad units on my pages. They recommend to add more ad units. It sounds like a bad joke after I lost 40% of my US traffic, possibly because of too many ads in the eyes of Google's search team.

Ugh. I had that e-mail too. It contains specific information (estimtated number of pages running < 3 ad units, a specific URL etc), but overall it's automatically generated.

Might be something that the AdSense team may want to review if it's indeed true that the ad-to-content ratio is one of the factors in the Panda update.

[BillyS]

Don't forget that the search team and AdSense team are at odds with each other most of the time, probably diametrically opposed in philosophy of what constitutes a "good site".

Go out to the website themadscientist found and look at some of the sites that were penalized. The ones I looked at were maxed out on Adsense AND they had other ad networks on there too. It was often difficult to find the information among all of the ads.
P-68

[Whitey]

Singhal: And based on that, we basically formed some definition of what could be considered low quality. In addition, we launched the Chrome Site Blocker [allowing users to specify sites they wanted blocked from their search results] earlier , and we didn’t use that data in this change. However, we compared and it was 84 percent overlap [between sites downloaded by the Chrome blocker and downgraded by the update]. So that said that we were in the right direction.

... interesting .... Chrome may gather speed in time as a contributor to editorial process'.

[Aaranged]

May interesting insights provided by the Wired article.

"Does this site have excessive ads?" While that was a question asked early, there's nothing to indicate that this wasn't incorporated into the algorithm modification they eventually rolled out. Tom Critchlow posted a lengthy comment [seomoz.org] at SEOmoz where he suggests that there's a possible correlation between the ratio of unique content to advertising above the fold and how much sites were impacted.

It goes without saying that there's a certain irony in this if its true, as the advertising carried by so many of the impacted sites is served by Google. While I'd hesitate to say that "the search team and AdSense team are at odds with each other" I can certainly verify - having worked on large content sites - that their is a very deliberately maintained wall between the two.

That they used quality raters in validating this update is no surprise, as this is a standard procedure. (I once encountered a Google quality rater at a party. Despite alternately plying him with beer and peppering him questions, I'm afraid I have nothing useful to share with you all.:)

[tedster]

the advertising carried by so many of the impacted sites is served by Google

Yes, that is an irony - and it speaks to the point that Adsense and Organic are more like two separate companies with a symbiotic relationship. Or in this case, conflicting goals too.

I like this article because it gives more insight into the process Google follows in going from a human idea to developing an algorithmic system - and then doing QA on the final result, too.

[js2k9]

Don't forget that the search team and AdSense team are at odds with each other most of the time

Good cop bad cop theater. They're not at odds, both serve shareholders.

[AlyssaS]

Google makes most of it's money from Adwords on the search pages, Adsense on content pages only brings in a fraction of the income.

They only make money from Adwords if searchers continue to trust and use their search engine. If people start to think they don't deliver the best search and go elsewhere, the main income gets threatened.

Sounds like they've taken a strategic decision to sacrifice some Adsense cows, to protect the Adwords golden cow. Which is smart, you have to hand it to them.

[walkman]

Freaking Google! They just sent out emails asking people to add more adsense units.

[Content_ed]

I think it's interesting that they ran a beauty contest rather than real testing. If they're going to get human beings involved, the questions to ask are:

Did you find what you were looking for?
Did it help you?

Asking whether or not a site is authoritative will get a majority "yes" answer if it features a photograph of an old white man in a white coat with a stethoscope.

As to too many ads, one is usually too many, but they probably mean the aesthetics again.

[incrediBILL]

Look at the big picture, they're not only pushing sites down into oblivion but are leveraging those same sites to make Google Places.

For instance, if you run a local directory, like I do, all of your data helps them pre-load Google Places and thanks for all the fish.

You want reviews of those places? You copy the review data as well, just like they're bullying Yelp and others with at the moment. It doesn't matter if you like Yelp or not, that's not the point. Google Places shows you all the review data you need to know to make a decision and too bad if the visitor never actually makes it to your site, sees the actual reviews, or pays your advertisers the time of day.

Google is indexing the web to become the web, I've said for ages they want to be a closed-loop environment by not just indexing the content, but making so much of it available onsite with cache and more you would never leave Google. Bing with their new decision engines and shopping tools are doing much of the same thing. Ecommerce of the future will be hosting your products in Google, Bing, Yahoo, Amazon or eBay forget running individual stores. We as webmasters are on notice, it's a short leash, as soon as they've figured out how to completely replace us, we're gone.

BTW, if you missed that last part, when the big players are the final hosts of ecommerce, another entire industry of ecommerce comparison sites and aggregators will also be extinct just like they're doing with local directories in this so-called farmers update.

Google is basically the internet version of Walmart, ready for some AdSense smart-priced rollback prices?

You know who's at fault for all of this?

WE ARE!

We allowed them to crawl without asking, cache our pages without asking, but they sent us tons of traffic without asking as well.

Guess what?

If it looks too good to be true, it probably is, and now yet another shoe has dropped.

Webmasters can take back the web.
We are the web.
Without us Google is nothing.
Freedom is within these 2 lines of a robots.txt file:

User-agent: googlebot
Disallow: /

Then take all incoming search traffic from Google and 301 redirect it straight to Bing to make a point.

'Nuff said.

[BillyS]

Freaking Google! They just sent out emails asking people to add more adsense units.

Did anyone look at these sites complaining about being downgraded? They don't just run the maximum number of Adsense ads, but they also run 10 additional ads from other networks too. On some of the sites I looked at, they have more ad text than content text.

We pretty much max out our Adsense ads too. But we also have cotent totaling 1,000 to 2,500 words on each page too. I suspect it's the ratio. Thin content, lots of ads. This is NOT a good user experience, which is what Matt has been telling people to create for years now.

I also think this move is very much aligned with what Google wants all around. Not only is this going to improve search, but it improves the Adwords program too because Google will get more impressions on higher quality sites.

[edited by: BillyS at 3:00 am (utc) on Mar 4, 2011]

[Freedom]

Sounds like they are looking at the word count to ad ratio.

Also, this story confirms to me what I've been picking up from many different places that they really went after health and medical sites.

[incrediBILL]

The part that roasts my rump is Google itself is aggressive with ads.

By the nature and intent of this update, shouldn't they just go offline?

[Bewenched]

Boy am I seeing tons and tons of Amazon doorway pages floating to the top for just basic searches it's unbelievable.

I'm not even looking for things in my own niche... sigh....

[Lapizuli]

It might be word count to ad count ratio. But most of my articles on third party sites that lost rank were in the 800-2000 word range, most over 1100 words. And not "wordy" - as in, they wouldn't have been identified as blah-blah-blah-beat-around-the-bush-to-say-nothing text even by an algorithm. They're clearly organized, too, not just meandering paragraphs.

Not saying this to defend my articles - I've been a writer since I was a kid, so whatever I write I always think is brilliant, then crap. :) Just trying to figure out if it's something simple, and I don't think so. The magic bullet isn't obvious.

[js2k9]

'Nuff said.

Indeed.

[browsee]

@incrediBILL, excellent words. Looks like Ad-sense is the major factor in this gigantic algo flop. I've decided to remove G ads from my site. I really don't care about the peanuts they send me every month. I can earn more money when the site is popular. 'G'ood riddance.

[edited by: browsee at 3:50 am (utc) on Mar 4, 2011]

[TheMadScientist]

Just trying to figure out if it's something simple, and I don't think so. The magic bullet isn't obvious.

In My Opinion: If you're looking for 'the one thing' that's the cause of the drop, it's the web page ... Don't worry too much about the source code, look at the page in the browser window ... The answer is right in front of you...

[TheMadScientist]

Oh, you might want to go find and remove all those orphaned pages you forgot about too, but for the most part, if you really want to know what the issue is, imo it's the page as a whole ... Sit there and look at it ... It's going to be WORK to fix.

[limoshawn]

If it looks too good to be true, it probably is, and now yet another shoe has dropped.

you're right on and the sad thing is that there's not going to be anything done about it until it's too late. well, nothing on a massive scale. i'm already getting ready for what i believe will be the next update, the next shoe, imo, is ecom. i see it coming next year and i'll call it the "wikipedia" update, the update that google will use to remove almost all ecom sites from their search results because:
[future google blog post]"We've heard from the people and the people don't want commercial websites in their search results. People have told our machines that they want the option of having commercial websites available but they want those websites to be labeled prominently. We are going to make sure that our users know the difference between commercial results and organic results by only showing commercial websites that match the users query in the section marked "ADS" that way when our users visit a website in the normal results they can rest assured that they will be visiting a wikipedia like, informational website"[/future google blog post]

[edited by: limoshawn at 3:38 am (utc) on Mar 4, 2011]

[indyank]

I agree with Bewenched.There are many doorway pages ranking on top these days. I guess this is one major loophole that is continuing to be gamed.

It does look like increased pageviews per visit is assigned greater weightage by this new algo, as an indicator for a "good site".Doorway pages are having a field day.

Smaller sites that provide what the user wants in a single page and right upfront suffer the most, as people are going to return quickly.

[buckworks]

Re "quality" .... I think I'm going to reread Zen and the Art of Motorcycle Maintenance on my trip to PubCon next week.

Here's what good ol' Wikipedia has to say about it:

[en.wikipedia.org...]

Phaedrus, a teacher of creative and technical writing at a small college, became engrossed in the question of what defines good writing, and what in general defines good, or "quality". His philosophical investigations eventually drove him insane, and he was subjected to electroshock treatment which permanently changed his personality.

I think some of us could identify with that! ;-)

[SevenCubed]

I sprinkle insanity on my breakfast cornflakes before I face the day.

[tedster]

Buckworks, that's AWESOME