Google "Quality Rater"

[Decius]

I was looking to hire someone for the purpose of linkbuilding and/or optimization and one reply came from India. This individual said he was hired by Google as a "Quality Rater" and his job was to browse the net for specific keywords and rate sites as irrelevent and relevent.

Given that this person is telling me the truth, does anyone else find it somewhat disconcerning that Google is now giving people the power to determine relevency in rankings?

Additionally, I'm not quite sure how a person who lives in a completely different environment such as India is equipped to validate or invalidate the relevency of search terms from a North American viewpoint.

Of course, he might be only doing Indian rankings, but regardless I can't say I feel comfortable with the concept that Google is permitting individual people to "choose" what they feel is relevant. I mean, the whole concept of popularity already makes top results stagnant... if popularity is extended beyond the algorithm then it becomes even more conformist.

[tedster]

This story started brewing a few years ago with the Google's Secret Evaluation Lab [webmasterworld.com] flap, and then last year with the Editorial Opinion Patent [webmasterworld.com].

Note that human-rated input is one part of an algorithmic ranking system, as decribed in the patent. These human editors do not sit there and directly create a SERP -- for India or anywhere else.

[edited by: tedster at 10:50 pm (utc) on July 9, 2008]

[Decius]

Certainly, but who's to say how heavily these ratings are counted. It's unlikely Google would invest the time and money training people and paying people to browse the internet and then value their opinions as much as they would 10 incoming links.

It's much more expensive to have human editing, and would therefore logically mean it creates a larger ripple than automated ranking works.

Perhaps it's a mechanism used by Google to compare sites and find patterns to improve the algorithm. If site A is voted as really good and B as spam, but the algorithm doesn't know how to tell the difference, then human intervention can help Google improve their algorithm.

Because in the end, there's no way they can use humans to clean up the SERPS. It has to be automated.

[tedster]

In the patent, the human editorial input is used in a very creative and algorithmic way.