Why Haven't Sites Come Back from Panda? Matt Cutts Tries to Explain

[walkman]

This is a rush(?) transcript from Dany Sullivan's blog so probably not everything is 100% correct. The italics and bolding are mine.
[searchengineland.com...]

DS: Talking about Panda, says that he’s getting a ton of emails from people who say that scraper sites are now outranking them after Panda.

MC: A guy on my team working on that issue. A change has been approved that should help with that issue. We’re continuing to iterate on Panda. The algorithm change originated in search quality, not the web spam team.
....
DS: Has it changed enough that some people have recovered? Or is it too soon?

MC: The general rule is to push stuff out and then find additional signals to help differentiate on the spectrum. We haven’t done any pushes that would directly pull things back. We have recomputed data that might have impacted some sites. There’s one change that might affect sites and pull things back.

DS: You guys made this post with 22 questions, but it sounds like you’re saying even if you’ve done that, it wouldn’t have helped yet?

MC: It could help as we recompute data. Matt goes on to say that Panda 2.2 has been approved but hasn’t rolled out yet.

DS: Reads an audience question – is site usability being considered as more of a factor?

MC: Panda isn’t directly targeted at usability, but it’s a key part of making a site that people like. Pay attention to it because it’s a good practice, not because Google says so.

Matt mentions 'pull back' but that's nonsense and very disingenuous of him. Pull back to me means letting a previously labeled bad content rank. We're talking about improved sites and content, no need to pull back, just reanalyze it.

So it's clear to me that this is a penalty. Maybe if you got links from every newspaper in the Northern Hemisphere you might escape but for the rest it looks like it depends on Google engineers. It took them 3+ months to admit it.

[freejung]

Yeah, I'm in the "get a[nother] job and maybe save the family from bankruptcy" category and I'm not liking it any more than you are. What I'm saying is, you always have a choice. What Google does with their algo is their choice. How you react to it is yours.

Particularly in this case we really don't know what if anything can be done to restore a site's rankings, so it would be particularly wise to react by doing things that are productive in themselves. Maybe getting another job falls into that category, and maybe redesigning your layout does too.

As for people who spent a lot of time spinning their wheels chasing the algo (making changes that do not improve their sites other than the attempt to recover from Panda), yeah, I feel bad for them but Google didn't force them to do that, it was their own choice.

[maximillianos]

Matt is right about one thing, CultofMac.com was not hit by Panda. Just look at Quantcast (direct measure data).

[HuskyPup]

I believe Matt said that if you made changes to your site, then they would be noticed the next time your site was crawled by googlebot and the index changes would take place as they normally do.

I can definitely confirm that I have seen improvements after tweaking a site, not changed it, simply SEO'd even tighter. I have not done this on any of my big sites but one of my niche-targetted .co.uk's and will test similar tweaks on one of my .asia sites soon.

Doing the same to my main B&M .com site is a huge task therefore I need to know I'm right before considering that!

[suggy]

It's amazing how people read what they want into stuff!

Then for the 'banging drums' it's a case of "click... whirrr... replay conspiracy theory/ anti-google vitriol...."

As I see it, this is what it means:-

1) "We have recomputed data that might have impacted some sites" -- in other words, they have iterated over the Panda data (you know, the data they said was updated periodically, not constantly) at least once since launch.

If you've made changes to your site, at least the part that Google has managed to index (maybe not all 10,000 pages you binned!) at the instance(s) they rerun the calcs is in the Panda data.

2) "There’s one change that might affect sites and pull things back" -- is saying that they have only back-tracked with one change.

Yes, sadly for you, Matt is not talking about your site (why would he be) he's talking about his algo and a change that softened it a little.

3) "It could help as we recompute data." -- a fuller response might have been: "look, we weren't anticipating such wholesale decimation of websites inflicted by spooked webmasters. Now we're way behind the current reality in terms of indexing all this and recalculating the link graph, etc. Frankly the index is now a mess of 404s, 410s, 301s, canonicals, noindexes and nofollows and it's going to take a while for the our picture of the web to catch-up with the new reality."

?!

[johnhh]

Fast becoming a Suggy fan :) or may be he thinks the same way as me

Frankly the index is now a mess of 404s, 410s, 301s, canonicals, noindexes and nofollows

I thought about this as I have made loads of changes . However - when I was considering them - is it right to have pages that say "no widgets available " , is it right to have old pages still indexed that have been superceded by new pages, is it right to have pages that have duplicates ?

Basically - have I been lazy and assumed our rankings wil go on for another 6 years ( we got hit in 2005 ) and not sorted this all out ?

We have pages returning in one section of our site that show a 1st June cache dates, but not all pages in that section, and I have identified why.

[dazzlindonna]

I've mentioned it before in another thread, but two of my company's websites were hit by Panda 2.0 and recovered with Panda 2.1 with no changes to the sites themselves.

True, it does seem as though those hit by Panda 2.0 had a chance to recover in 2.1. It seems to be much less likely for those hit by Panda 1.0 for some reason.

[superclown2]

: "look, we weren't anticipating such wholesale decimation of websites inflicted by spooked webmasters. Now we're way behind the current reality in terms of indexing all this and recalculating the link graph, etc. Frankly the index is now a mess of 404s, 410s, 301s, canonicals, noindexes and nofollows and it's going to take a while for the our picture of the web to catch-up with the new reality."

With the computing power Google have got they could download the whole web and sort it in an afternoon.

Panda was a completely new ranking system with hundreds of new variables. It was inevitable that there would be bugs.

When Google run tests they use their employees as guinea pigs. Problem is that highly educated geeks from, mainly, wealthy backgrounds don't do searches like the rest of us. This system had to be released onto the public to test it properly and now ironing out those bugs will be a priority but it all takes time and experimentation.

[walkman]

If you've made changes to your site, at least the part that Google has managed to index (maybe not all 10,000 pages you binned!) at the instance(s) they rerun the calcs is in the Panda data.

I assure you I didn't have 10,000 pages or even close, but even now I have 4 times less than I had on Panda day. And Google gets them every 2-3 days, 100% of them. On my non-pandalized sites I have over 100,000 in one and 30k+ on another, traffic is way up there. Whether content is analyzed directly or not Matt answers is very clearly if you read between the lines:

DS: You guys made this post with 22 questions, but it sounds like you’re saying even if you’ve done that, it wouldn’t have helped yet?

MC: It could help as we recompute data.

He speaks of 'data,' not 'as we re-index and re-process the content of the page.'

"There’s one change that might affect sites and pull things back" -- is saying that they have only back-tracked with one change.

Or if you get 4800 links from different newspaper sites. We don't know what that signal is. But we can ask around and see if any Feb pandalized site had come back or not.

3) "It could help as we recompute data." -- a fuller response might have been: "look, we weren't anticipating such wholesale decimation of websites inflicted by spooked webmasters. Now we're way behind the current reality in terms of indexing all this and recalculating the link graph, etc. Frankly the index is now a mess of 404s, 410s, 301s, canonicals, noindexes and nofollows and it's going to take a while for the our picture of the web to catch-up with the new reality."

Google is talking about figuring out why mangos get hot in boxes, so it's obvious they have more than enough computer power for that. My site:domain.com and all WMT data has been accurate for well over a month.

Rlange, two of my competitors that were hit by Panda 2 are showing a major increase by Alexa for the past month. It corresponds with the loss they suffered. As far as I can tell, no changes were made by those sites, and at least one truly sucks. I know it's Alexa but presumably they used the same method before and after.

[gadget26]

Gentlefolk, as a newbie just trying to keep my Pandalized 1.0 business from going down the last swirly-gurgle-bit of life's commode, I REALLY NEED (and appreciate enough to become a paid member) the kind of speculation that I find here. Purely negative posts don't really help me. Please keep the speculation going. The truth is in there somewhere.

My 2 cents.

[tedster]

I'm all for intelligent theories, too. But let's keep the editorializing out of it so that the theories have a chance to pop off the page - thanks.

[netmeg]

(there's no litmus test for senior members - otherwise I wouldn't be one)

But yea. I'm pretty tired of opinions being presented as "clearly" facts as well.

[tangor]

All the fixes mean nothing until one knows what to fix! One or two things that MC said early on made sense (thin, content farm) but all the REST which happened to many webmasters everywhere, did not, does not, and will not make sense until we are (not likely) told what new parameters were incorporated with Panda x.x.

Meanwhile, diversify!

[edited by: tangor at 1:54 am (utc) on Jun 9, 2011]

[Leosghost]

You are not alone in that feeling netmeg, not by any means.

[AlyssaS]

All the fixes mean nothing until one knows what to fix!

Well one big clue came from Google on their live webmaster chat - see this article for the report:

[searchengineland.com...]

And the interesting bit was this (it's a response to someone asking why it has not been rolled out internationally):

“There were some characteristics that were more applicable to English-language sites,” Cutts said. The original question came from a viewer in Poland, and Cutts explained that “the link structure of websites in Poland is a lot different” from the link structure of sites in other countries.

Panda seems to me to be entirely about on-page stuff. Therefore "link structure" is referring to internal links. Go and look at some of the Pandalised sites - they have a really heavy link structure.

I don't think it's anything as simple as internal links are bad - more like they are applying ratios of some sort and some sites have tipped onto the wrong side of Panda.

I've been banging on about Hubpages v Squidoo since late Feb [webmasterworld.com] - but though these sites are very similar (both have spam, both user generated content, both have a lot of ads) - the one difference is that Squidoo has a light internal link structure and Hubpages is really really aggressive with their internal linking.

But sorting it out isn't simple. Suppose you remove those links - the internal links might have been the only things supporting some pages, possibly a lot of pages. So you might come back a bit, but not really back to where you were before Panda where your internal link structure was supporting everything. You are going to have to get new external links to those pages which have lost internal link support, or resign yourself to not getting traffic back for those pages at all.

For smaller sites - I think someone reported getting their rankings back simply by removing links to archive pages. And I've heard some reports of people regaining rankings by removing excessive tags. It's possible those archive pages and tags didn't do much anyway, so no loss in removing links to them. But for more complicated sites, it might be another matter altogether.

And of course there are other elements to this algo as well, such as ad-to-content ratio, duplicate content and so on. If you get a black mark against too many elements, you won't recover till you've removed them all.

[tedster]

did not, does not, and will not make sense until we are (not likely) told what new parameters were incorporated with Panda x.x.

Here's how I see it.

The Panda algorithm is based on machine-learning. This means it's a predictive algorithm, assembled by an automated process. It's predictive because it works from a "seed set" that was generated by human judgment. The machine learning program is let loose across a huge pile of factors to discover what data might predict "shallow quality", as defined by the seed set. The prediction will not be accurate in the case of every website, but as the process iterates it does become more and more accurate.

When the machine predictions look good and their results pass some human QA, then those factors it identified, however they are weighted and combined, become the algorithm. This stays in place until that entire process can be re-run and generate a new version of the algorithm, incorporating new factors. As I understand it, that's the "running the data" part of Matt's comments.

The full list of parameters at any one time is likely to be even more confusing to the general public than the current situation is. And for a select few people, that list would open the door to gaming the algorithm.

So I'd say you're right - we're not going to get the recipe. If we did, we might be astounded at some of the data that Google is maintaining. I'm sure there a lot more than we've ever guessed, and much that they collect but have never used before.

[supercyberbob]

Who do I have to pay to get this list of parameters?

[tangor]

Who do I have to pay to get this list of parameters?

In the not-so-tinfoil-hat world of biz (and advertising) you might have uncovered the REAL direction of Panda... a pay to play kind of serp... (I know, I rant this from time to time... but that's the only thing that makes SENSE!)

Follow the money...

[AlyssaS]

P.S. Small anecdote about a site that got taken out in Panda 2.1 and came back yesterday. It's a small site, one of my earliest, and I hadn't set my spam traps properly, so it got over-run in the comments (each post with about 160 comments of spam). I zapped them all and expected the site to come back when re-crawled, but nothing happened - till yesterday when the re-gen happened.

Why did it need a panda re-gen to happen rather than a simple re-crawl? All I can think of is a) perhaps panda is looking for certain types of phrases it associates with spam, and once you are marked down it needs a panda re-gen to lift the blackmark or b) the links in the comments triggered the down-grade - i.e. the bot looked at the html and saw too many a tags, it tripped a ratio and down I went. And then got restored on the re-run.

[tedster]

nothing happened - till yesterday when the re-gen happened.

But according to Matt, version 2.2 of Panda is not yet live - right? So something else must be in the mix.

How long ago did you dump all the spam - how many crawls of those pages, roughly at any rate?

[AlyssaS]

According to my log I found and zapped the spam on 19th May (the site had dropped in rankings on 9th may which is consistent with panda 2.1). It got crawled a few days later. And I kept re-checking the cache and the cache date kept getting updated.

I think there was a re-gen of sorts starting late on 6th Jun, it's consistent with the serps upheaval reported in many places. But of course I might be completely wrong and the delay in the site getting back was something else.

But zapping the spam was all I changed - I didn't add any content or amend the structure in any way.

[plondon]

Thanks guys, I found this thread extremely reassuring. I've only been an Adsense publisher for 9 months, and in that time I experienced a steady climb in traffic and income. Until early May when I lost about 2/3rds of both in a week. Through May I steadily climbed back to about 2/3rd of my April stats: beginning to average about £13-£15 per day...

Then something happened again in June:
Wed 1st: Page views: 488 - Est Earnings: £11.05 - CTR: 6.15%
Thu 2nd: Page views: 417 - Est Earnings: £16.74 - CTR: 8.63%
Fri 3rd: Page views: 389 - Est Earnings: £11.76 - CTR: 7.71%
Sat 4th: Page views: 308 - Est Earnings: £04.20 - CTR: 5.84%
Sun 5th: Page views: 337 - Est Earnings: £11.51 - CTR: 8.01%
Mon 6th: Page views: 493 - Est Earnings: £28.91 - CTR: 8.11%
Tue 7th: Page views: 378 - Est Earnings: £05.19 - CTR: 4.23%
Wed 8th: Page views: 315 - Est Earnings: £01.46 - CTR: 2.22%

Yesterday was my worst day since I began. Beating even the first week of May. But Monday was my best day ever in earnings, although my best traffic day was around 690 in late April.
(Saturdays always drop. But midweek is traditionally my best time.)

These stats come from about 10 performing sites, with between 10-50 pages on each. All domains about 7 months old. All original content. Mainly US audience.

All I've done since beginning of May is begin adding content more regularly, about every 5 days. And continued link building low-medium PR sites through commenting, and article marketing. Plus I upped my RSS feed distribution. Nothing particularly 'shadey' and certainly nothing that would raise any flags. These are small sites, small niches and I'm a one-man show working from home.

Anyone make any sense of this?
Why is the Panda picking on me?

[rlange]

AlyssaS wrote:
I think there was a re-gen of sorts starting late on 6th Jun, it's consistent with the serps upheaval reported in many places. But of course I might be completely wrong and the delay in the site getting back was something else.

Something happened, definitely. My company's main site saw a significant increase in traffic starting Monday (June 6th).

The odd thing, though, is that no changes were made to the site and the new traffic is all going to a different section of the site than the traffic that was lost.

If I had made changes I'd agree that it was a simple re-run of the algorithm, but since I didn't change anything I can only think of three possibilities: 1) Google changed something on their end, 2) some external signal for our site changed dramatically, or 3) the algorithm can produce [significantly] different results each time it's run.

--
Ryan

[HuskyPup]

Panda seems to me to be entirely about on-page stuff.

Insofar as my sites are concerned I am seeing the same literally from the titlebar, meta description and all the way down the page with h1, image alt and title etc.

I'm adding extra text content IF available but otherwise tightening-up all my own my SEO stuff and if it means generating a new page because the Panada does not understand that keyword1keyword2 for me is the same as keyword2keyword1, then so be it.

So far this is working well in the UK SERPs.

[Planet13]

And continued link building low-medium PR sites through commenting, and article marketing.

Anyone make any sense of this?
Why is the Panda picking on me?

The general consensus is that the types of links you are building have seen reduced value, and will continue to see declining value from google as it moves forward.

Also, since these kinds of links are relatively easy to obtain, it is VERY likely that your competitors are all doing the same type of link building.

[Planet13]

@ tedster

This stays in place until that entire process can be re-run and generate a new version of the algorithm, incorporating new factors.

But according to Matt, version 2.2 of Panda is not yet live - right? So something else must be in the mix.

I apologize if I misinterpreted your comments. I am a little confused by this, so maybe you can clarify a bit.

Are you saying that one's rankings are effectively "locked" until each new version of the algorithm is made (i.e., each time Panda is "run")?

Are you saying that sites can't have their rankings reassessed under Panda by simply being re-crawled / re-calculated under the CURRENT version of the Panda alg?

While I can understand your points about the learning nature of the alg requiring it to be re-run (so that the algorithm - not the rankings - can be adjusted), I would still thank that ranking adjustments would take place before it is re-run.

[tedster]

I am saying the FACTORS that the algorithm uses are locked in, not individual rankings. There's plenty of evidence that websites can change their rankings, even during the same version of Panda.

In the case that AlyssaS described, it sounds to me like some non-Panda part of the total algorithm was at work.

[Shaddows]

Think of it in 3 stages.
1) Definition of criteria that is being considered. This consitutes the Panda version. It is defined by engineers. Example: Keyword density (AFAIK, KW density is NOT a Panda factor)

2) Current "Understanding" of criteria. This is machine learnt, iteratively. There are several iterations between version updates. It is defined by seed sets, and refined by world data. Example "Optimum KWD = 12%, ranking points allocated on a power curve" (12% is NOT a good KWD, and a power curve is NOT a good distribution curve for this data)

3) Application to crawl data. This is updated on the fly like regular data. Example "This site has a KWD of 15%, so gets 10 points"

Edit for clarification

[freejung]

3) Application to crawl data. This is updated on the fly like regular data.

That makes sense conceptually, but I'm not sure it explains what we're seeing. Under the normal algo (which is static until manually updated, presumably) if you make changes to your site that would impact your ranking, you see changes in ranking quickly (admittedly you don't normally see the full effect immediately, but you generally see some effect).

Now people are reporting no change in ranking even after making radical changes to their sites, changes that should have had some effect even under the old algo. Why? I don't think we've explained that.

Under the theory you propose, if you do nothing to your site your ranking will only change when the algo is re-gen'd. But your ranking would change, under your theory, if you make significant changes to your site that impact the factors Panda is measuring. If that's the case, why do so many people report that their Pandalized rankings are remarkably stable even after making radical changes?

Maybe we're just not changing the right things.

[HuskyPup]

Maybe we're just not changing the right things.

From what I am seeing, yes, that would seem correct, I have been very clinical about my changes, I've analysed in-depth every page I've changed and whether I felt it would be better from an informational and SEO perspective, whether it would work or even be practical for other sites with thousands of pages I have no idea but it is working for a couple of my small sites right now.

[tedster]

Under the normal algo (which is static until manually updated, presumably)

My understanding is that Google's entire algorithm is now dependent on machine learning. Human quality raters give feedback on various SERPs and that creates seed sets of less than optimal rankings. Then the machine-learning program ranges over the data signals that Google collects to learn how those factors could be applied to create a more optimal SERP. When the new machine-proposed algo passes some quality checks, it becomes live.

why do so many people report that their Pandalized rankings are remarkably stable even after making radical changes?

And that is a mystery - although many people have reported incremental gains in traffic during those interim periods.

The challenge we face is being limited to anecdotal reports, for the most part. It's hard to arrive at anything rigorous that way. I know I certainly don't have a big pile of Pandalyzed sites to work on in a scientific way.