Forum Moderators: Robert Charlton & goodroi
Q: How do they tell if they have bad results?
A: ...they have 10,000 human evaluators who are always manually checking the relevance of various results.Article by attendee Dare Obasanjo [25hoursaday.com]
Next time we're trying to figure out some odd change in the SERPs, we might do well do remember this human factor, eh?
lets be honest anything apart from search has been a huge, huge, huge (revenue) failure for Google.
That sounds like a topic for the GOOG forum, not the Google Search News forum, but in any case, the statement is completely wrong, as one can see by reading any Google earnings report.
It's just a matter of improving organisation.
So we need to believe that G will get progressively more serious about content and linking technique control. When this is complete few sites will remain in their positions , by links alone and content alone. What will evolve are techniques relying solely on background data comprising such comparative things as adherence to guidelines and user activity measurements of quality.
These types of things are the new voting paramaters for a website. I think links will become a thing of the past, and I recall some recent posts indicating these thoughts to.
This does mean that G will move to a new level of complete dominance IMO
These types of things are the new voting paramaters for a website. I think links will become a thing of the past, and I recall some recent posts indicating these thoughts to.
I don't think links will become a thing of the past with Google; they just won't be as powerful by themselves as they used to be, and Google will continue to improve its ability to distinguish between valuable and meaningless links.
I think those human evaluators have a say on the ranking of the top few hundred sites..
You could pretty well kill the bias rate by recycling the same SERP to a variety of reviewers, wisdom of crowd type of thing.
they pay around $12 - $15 an hour, work from home, part time, no benefits. the people are not highly specialized, just trained to look for obvious problems, and rate sites.
Thanks for feedback.
they pay around $12 - $15 an hour, work from home, part time, no benefits. the people are not highly specialized, just trained to look for obvious problems, and rate sites.
And that might explain the poor quality of Google's serps that have been mentioned on different threads on WebmasterWorld for the latest years.
We know that multi billion Google could afford to spend more to improve its serps quality but they haven't unfortunately.
IMO, better algos driven serps than the $12 - $15 evaluators you and another friends have mentioned on this thread.
Talking about the human-role-in-Google :-)
I'm sure that $12 - $15 is top money also in Mogadishu in Somalia for "work from home, part time, no benefits. the people are not highly specialized, just trained to look for obvious problems, and rate sites."
Oh well...you get what you pay for :-)
For me the role of human in Google is better explained in the great efforts of kind Googlers like Matt Cutts, Adam Lasnik and Vanessa Fox to communicate with the webmasters communities on forums and during conferences. Power to Matt, Adam and Vanessa!
It was mentioned earier about the Google Toolbar, could it simply be that they have 10,000 active users who use the vote buttons?
Possibly the human evaluators work is related to Random Query Evaluation [google.com].
Google needs to maintain its aura of supreme ad context management to maintain its market position. To rely on human editors, after all, is an admission that Google is just another Yahoo or DMOZ with a different story line. They wouldn't want that now, would they?
To rely on human editors, after all, is an admission that Google is just another Yahoo or DMOZ with a different story line.
Fortunately, they'll never have to make such an admission, since they don't "rely on human editors."
Here is [mattcutts.com] what some Googlers said:
- Marissa Mayer : “Up to today we have relied on automation, but I believe the future will be a blend of both, combing the scale of automation and human intelligence.”
- Larry Page: "Larry says search is finding content… and that Wikipedia found a better way to organize information. he seems to like the model of using humans and process and machines."
- Matt Cutts: "So I think too many people get hung up on “Google having algorithms.” They miss the larger picture, which (to me) is to pursue approaches that are scalable and robust, even if that implies a human side. There’s nothing inherently wrong with using contributions from people–you just have to bear in mind the limitations of that data."
- Udi Manber said [stonetemple.com]: "If somebody says they don’t like a particular search result that’s a signal. So, we’ve been using that for a long time, and we are working on new ways of using it."
Its therefore of importance to know exactly which 10.000 human evaluators Marissa Mayer was referring to in her session with Dare Obasanjo. Was she referring to Google internal employees or the $10 - $15 temp evaluators worldwide?
[edited by: reseller at 8:56 pm (utc) on July 11, 2007]
From what I read, it seems Google is moving from algorithms-driven-search engine to the direction of semi-human-edited-search engine.
You're reading your own preconceptions into what was actually said. There's no way that a "semi-human-edited" search engine would be "scalable," to use a popular Google term. Where human input can be valuable is in providing cues and clues for writing and/or training the software that does the work. (And, of course, Google has been doing that all along; PageRank is based on human input, and it's highly unlikely that Larry Page and Sergey Brin let their original algorithm write itself back in the late 1990s.)
ADDENDUM: The proof is in the pudding, and having paid evaluators to help identify spam, distinguish between good and bad results, etc. should improve Google's search results over the long term--regardless of where they're being recruited, how much they're being paid, and whether their input is being used to train software, to facilitate research, or (less likely) to simply whack pages that don't deserve to be in Google's SERPs.
..and having paid evaluators to help identify spam, distinguish between good and bad results, etc. should improve Google's search results over the long term--regardless of where they're being recruited, how much they're being paid,
Its not only how much they are being paid which worry me. Its a statement like this one [webmasterworld.com] too:
"they pay around $12 - $15 an hour, work from home, part time, no benefits. the people are not highly specialized, just trained to look for obvious problems, and rate sites."
Would any site owner and webmaster feel comfortable knowing that such kind of $12 - $15 evaluators might affect directly or indirectly which site to stay and which site to be deindexed on Google index?
Would any site owner and webmaster feel comfortable knowing that such kind of $12 - $15 evaluators might affect directly or indirectly which site to stay and which site to be deindexed on Google index?
I imagine that depends, to a great extent, on whether the site owner or Webmaster has anything to hide. :-)
