Google Publishes 160-Page Search Quality Rater Guidelines

[netmeg]

---

System: The following message was cut out of thread at: https://www.webmasterworld.com/google/4778146.htm [webmasterworld.com] by engine - 1:00 pm on Nov 20, 2015 (utc 0)

---

Rustybrick has posted a link on searchengineland to download the full Guide here: [searchengineland.com...]
And here's the Google post about it (added later) [googlewebmastercentral.blogspot.com...]

PDF is here: [static.googleusercontent.com...]

[edited by: Brett_Tabke at 5:37 pm (utc) on Nov 20, 2015]
[edit reason] added link to PDF [/edit]

[engine]

Now that Google has decided to publish its Search Quality Rating Guidelines document online we can all read it and absorb the content. It's a 160-page PDF file, so get ready for some coffee-time reading.

Developing algorithmic changes to search involves a process of experimentation. Part of that experimentation is having evaluators—people who assess the quality of Google’s search results—give us feedback on our experiments. Ratings from evaluators do not determine individual site rankings, but are used help us understand our experiments. The evaluators base their ratings on guidelines we give them; the guidelines reflect what Google thinks search users want. Google Publishes 160-Page Search Quality Rater Guidelines [googlewebmastercentral.blogspot.com]

Link to the 160-page Google Search Quality Rating guidelines (PDF) [googlewebmastercentral.blogspot.co.uk...]

Earlier post Google Guidelines - October 2015 Mobile Edition [webmasterworld.com]

[ken_b]

I looked at the guide last night, but apparently you need to be logged into your Google account to follow links in the pdf. Anyone else see that?

[engine]

I haven't yet tried the links. I can only assume it's way of tracking who's seen it. I don't believe it's anything sinister, just normal tracking for a publications that would normally be an in-house document.

[superclown2]

An interesting read. However I still see sites at or near the top for valuable keyphrases that break all those rules. They are invariably only a few months old and have thousands of links from forums or spun content sites. This isn't something that has just happened, I've been watching some of them climb the SERPs for months.

Theory and fact are often completely different.

[Kelowna]

If these guidelines actually worked for getting pages to rank then they would never have been made public.

Time would be better spent researching the sites ranking above you than reading google's nonsense.

There is a reason that they stopped updating the toolbar pr. They hide what works and try to confuse the masses with their 160 page guidelines nonsense.

I will give you the short version of the real quality guidelines. . LINKS WORK - GET SOME

[EditorialGuy]

If these guidelines actually worked for getting pages to rank then they would never have been made public.

They aren't SEO guidelines. They're guidelines for quality raters to use as part of Google's QC process.

[7_Driver]

If these guidelines actually worked for getting pages to rank then they would never have been made public.

They're not intended to be SEO guidelines. They're used by the human quality raters to determine quality web pages from non-quality ones. And their scores are used to train parts of the algorithm and/or evaluate the success of algorithm changes. So they're an excellent guide to what Google is trying to get the algorithm to do in future.

Obviously they're mainly concerned with on-page factors - but there's some off-page authority stuff in there as well.

Lots of interesting quotes in there, like:

Remember that high quality content is defined as content that takes time, effort, expertise, and talent/skill.

The presence or absence of Ads is not by itself a reason for a High or Low quality rating.

We do not consider legitimately licensed or syndicated content to be “copied”

and lots more...

It's great they've released these - since we can now click through and see the examples - which we couldn't do with the leaked versions in previous years - they're very interesting and well worth checking out.

[seoskunk]

I will give you the short version of the real quality guidelines. . LINKS WORK - GET SOME

They never stopped being a link focused engine.

[aristotle]

Links are what Larry and Sergey started with, the original heart of the algorithm and the key to Google's early success. Obviously they don't work as well now because of the long ongoing war with spammers and SEO practitioners. But links have such a hallowed place in the company's history that most likely every effort will be made to keep them as a major ranking factor.

[EditorialGuy]

Forget about links in the context of this discussion. We aren't talking about PageRank and other algorithmic factors that help pages rank high in Google Search results, we're talking about the kinds of pages that Google wants to see ranking high in its search results.

[Walt Hartwell]

EG,
Since you appear to have the inside knowledge of what Google would like to see ranking high in it's search results, perhaps you can share "the kinds of pages that Google wants"
My specific interest is how long can an Aardvark live in an alpine environment.

Or, are you just saying Google likes some things about pages, but you are sworn to secrecy and can't divulge the knowledge imparted to you?

[superclown2]

They aren't SEO guidelines. They're guidelines for quality raters to use as part of Google's QC process.

The specification ideals represent what Google claims to be high quality sites that should be high in the rankings. But they are not. The top rankings go to big brand sites and mega-spammers with masses of links.

Like I said, theory and fact are two different things. I am sure that Google could easily make high quality sites into top rankers - but they don't. Therefore this document is just either PR, or FUD.

[EditorialGuy]

EG,
Since you appear to have the inside knowledge of what Google would like to see ranking high in it's search results, perhaps you can share "the kinds of pages that Google wants"

No need for "inside knowledge." The guidelines are quite clear.

[SEOPTI]

Without even looking at this paper because I disgust their monopoly, is it a solution to their nasty Panda in this?

[netmeg]

One more time:

They aren't SEO guidelines. They're guidelines for quality raters to use as part of Google's QC process.

Moreover, why should anyone else do your homework for you? If you want to know if there is something useful for you in the document, then you should read it yourself.

[glakes]

Are these quality guidelines enforced by offshored help whom have a limited concept of the English language and fractured understanding of the sites which they are charged to evaluate as was the case in the past?

[Walt Hartwell]

EG,

Many years ago someone on a different forum posted a comment about how you can't argue with an idiot. You and fathom on this forum both have your little "concepts of the internet", which I fully agree you are entitled to. But, you are both such broken record, repeating the same limited viewpoint that it is painful to try to read your "commentaries".

I don't say this as a flaming type thing, I think you both could actually contribute to the general knowledge pool. So why not add to the knowledge rather than doing fanboi responses?

[superclown2]

Are these quality guidelines enforced by offshored help whom have a limited concept of the English language and fractured understanding of the sites which they are charged to evaluate as was the case in the past?

In the past I saw innumerable excellent websites destroyed after visits from G employees who may well have matched this description. I suspect - and this is theory only - that their job was to flag sites they didn't like the look of so that more senior people at Mountain View could have the final decision. I'm not seeing it now which in some ways is a good thing but I'm also seeing blatant spam sites with zillions of spun links sitting at or near the top for months. Google's quality control has not improved since Matt Cutts left and I doubt if this document will help one jot.

Sure, it contains guidelines that we should all follow in an ideal world but this is the real one and we all have to put bread on the table. Treat it as though it's radioactive is my advice. In other words handle with extreme care.

[superclown2]

I think you both could actually contribute to the general knowledge pool.

If we all agreed on everything there would be no point in this forum. We can all learn something, even from those we profoundly disagree with. I think that this document is worthless FUD, others believe that there is valuable knowledge in there. Perhaps the answer is somewhere in between. Certainly from an altruistic point of view the quality guidelines, if we all followed them, would probably result in a better Internet. The problem is that Google themselves don't follow them.

[netmeg]

Are these quality guidelines enforced by offshored help whom have a limited concept of the English language and fractured understanding of the sites which they are charged to evaluate as was the case in the past?

Since I personally know a handful of ex quality raters who are native English speakers and writers, and of American and Canadian origin, I'm thinking maybe not. But perhaps you have some information the rest of us don't.

(Just wow)

[EditorialGuy]

I don't know why anyone should be called a "fanboi" for pointing out the obvious: that a detailed Google handbook for quality raters titled "Search Quality Evaluators Guidelines" is likely to be a pretty good indicator of the kinds of pages that Google likes to see rank.

Such a document doesn't help SEOs sell their services, of course. It might even lead some clients or prospects to decide that their money could be spent more productively on editors, writers, and designers.

[randle]

It might even lead some clients or prospects to decide that their money could be spent more productively on editors, writers, and designers.

In todays search climate, that would be where their money would produce the best return on their investment. Companies really need to start thinking in much longer terms than what traditional paid "SEO" services have been sold on.

The bottom line is its really hard to get poor sites to rank well, no matter what the latest seo flavor of the month is. Ten years ago, even 5, not so hard - different game today. Quality platform, design, mechanics and building a solid brand is really the best road - even if it takes a year or so to get there.

[aristotle]

But does this rating process take account of the accuracy and intrinsic value of a site's content?. For example, over the years we've had many discussions here about "authority" and how it might be evaluated, or how Google might evaluate it.

Suppose that the world's foremost expert on antique widgets creates a comprehensive informative website about this subject, but doesn't follow some of the quality guidelines that these raters use. If a rater judges the site by these guidelines, rather than by the value and usefulness of its content, then the site could get lower marks than it deserves.

[EditorialGuy]

Suppose that the world's foremost expert on antique widgets creates a comprehensive informative website about this subject, but doesn't follow some of the quality guidelines that these raters use.

The guidelines are for Google's quality raters, not for site owners. For site owners, they're useful to the extent that they show what kinds of things Google is looking for.

One of those things (described at length in section 2.7) is reputation. I'd imagine that "the world's foremost expert on antique widgets" would rank high for reputation even if he or she had never heard of Google's guidelines for quality raters.

[netmeg]

Suppose that the world's foremost expert on antique widgets creates a comprehensive informative website about this subject, but doesn't follow some of the quality guidelines that these raters use. If a rater judges the site by these guidelines, rather than by the value and usefulness of its content, then the site could get lower marks than it deserves.

Google can't rate every business/organization/entity on the planet, and I sure don't think we want it to. All they can really do is guess at user experience and whether or not the users are happy with the WEBSITE - not the business itself (or other entity).

IMHO this has been one of the biggest misunderstandings when it comes to how Google search works. It's just not enough to be a stand up guy.

[fathom]

Many years ago someone on a different forum posted a comment about how you can't argue with an idiot. You and fathom on this forum both have your little "concepts of the internet", which I fully agree you are entitled to. But, you are both such broken record, repeating the same limited viewpoint that it is painful to try to read your "commentaries".

I don't say this as a flaming type thing, I think you both could actually contribute to the general knowledge pool. So why not add to the knowledge rather than doing fanboi responses?

I am truly honored that you find enough rational to bring me up in a thread I never participated in. Thanks fanboi!

[FranticFish]

I'm amazed that this is the best system Google think they can come up with. If I wanted to use humans to try to discover quality on the web, here's what I'd do:

1) Select prospectives raters from among Google account holders who have only GMail accounts / personal G+ accounts (so no Analytics, WMT etc - no professionals)
You could create all sorts of demographics based on interests / education level / other factors as determined by web history and search history, G+ profile, even Facebook profile (I believe the two companies share information).

2) Approach people via Google account (flashing message when logged in, stickymail in GMail etc)

3) Rater agrees to install app on their device(s) that will ask them to complete a short survey when certain things happen e.g.
- they want to say that they are particularly happy or unhappy with a page they've visited;
- they search, navigate to a page and return to Google within a short space of time;
- they navigate to a page and leave within a short time, or return to the previous page.

4) Rater's browsing and search habits monitored to make sure that they don't vary significantly after becoming a rater from the time before they were a rater.

Google instead decided to:
- issue guidelines to try to tell people what quality is (rather than let them define it in their own terms by their actions);
- get those people to rate pages returned as results for keywords they don't choose (rather than let then use the web as they normally would); then
- try to interpret those people's impressions themselves.
Am I the only one that thinks that this is (a) very circular and (b) has a lot of room for error?

[EditorialGuy]

Actually, I think Google's approach makes perfect sense, given the purpose of the quality ratings and the amount of work that's required of the raters. I also suspect that Google has had quite a bit of internal discussion about the best ways to gather data for QC purposes.

Something else to keep in mind: Public-opinion surveys, which are what FF seems to be advocating, typically have extremely low response rates. (A recent article in THE NEW YORKER said average response rates for public-opinion polls are in the single digits these days.) So, even if Google wanted to go that route, it would have to make a lot of guesses and assumptions (as other pollsters do) to compensate for variations in response rates for different types of users. As the expression goes, "Garbage in, garbage out."

[FranticFish]

I didn't say it wouldn't be paid, and Google has the biggest group of users in the world.