What worries me is the question if those evaluators are civilians (non owners of web sites)
I'd hate to compete in the serps with one of those with all the inside information and connections they have, sort of like dmoz all over again.
I don't get the impression, from public statements made by GoogleGuy and others, that Google's quality evaluators are being used to "clean out" search results. There's no reason to believe that they have the power of DMOZ editors, who act as gatekeepers for their assigned DMOZ topics.
|Even if they do have 10,000 quality evaluators, a lot of those people may be (and probably should be) part-timers. |
I think most of them don't even know they are considered human evaluators by Google... ;)
[edited by: wildbest at 5:46 pm (utc) on July 10, 2007]
Do these 10,000 evaluators have the power to make manual serp changes? If so, that is scary.
|sort of like dmoz all over again |
My thoughts exactly.
|Do these 10,000 evaluators have the power to make manual serp changes? |
Not according to the patent [webmasterworld.com]. You're right, that would be a scary scenario.
|Any guesses as to the salary? |
According to this article [searchbistro.com]:
|The Google testers are paid $10 - $20 for each hour they filter the results of Google. Payment is done through Payroll. The international agents are instructed by the Human Quality Evaluation Team of Google. |
Nobody in this thread has mentioned a visit to their website by a human evaluator.
I did have visits from them on a new site something like 2-3-4 months ago. I spotted them as I was monitoring the logs of the new site. I recall seeing something like 5 visits from various computers based in my country, and the referrer contained something like G evaluation.
My country (former Soviet bloc) is not a big one, population is about 1/30 of that of the US. If my local language site could receive 5 evaluators (probably not everyone in G's local team visited my site), the number of staff in larger markets might be magnitudes larger.
If I remember correctly, I was paid around $15/hr.
To answer the question about being able to change SERPS, that was not an option.. =)
We were told our data would be used for research purposes and would not directly affect the search results.
It is my opinion that our rating data was/is used to help better tweak their algorithm.
Did you get the job at that time through a similar job announcement like this one?
|QUALITY RATER - (SPANISH, DUTCH, ITALIAN, FRENCH) This is a temporary role offered through Kelly Services. Google Inc. is recruiting part-time, temporary, home-based workers to help with work on a search quality evaluation on a project basis. You would work at your own pace, and the time and length of any particular work session would be up to you. Candidates will evaluate search results and rate their relevance.... |
< I shortened the quote
see Terms of Service [webmasterworld.com] >
[edited by: tedster at 7:49 pm (utc) on July 10, 2007]
Google employs over 10,000 full time people and has temporary employees as well. I would consider every one of them a "human evaluator" that has an impact on search results.
|Did you get the job at that time through a similar job announcement like this one? |
reseller - Yes, that look pretty much exactly like the job ad I responded to.
>>Google employs over 10,000 full time people and has temporary employees as well. I would consider every one of them a "human evaluator" that has an impact on search results.
They don't want to listen to us kkobashi. I think you, Brett, and I are all on the same page.
Thanks for feedback. Much appreciated.
But it seems they weren't interested at all whether the Raters themselves are webmasters owning for example business sites or are part of affiliate programs. I mean Google didn't care about the objectivity (fairness) of the Raters.
|But it seems they weren't interested at all whether the Raters themselves are webmasters owning for example business sites or are part of affiliate programs. I mean Google didn't care about the objectivity (fairness) of the Raters. |
I did have to jump through hoops to get the job since I was a webmaster. I had to get some sort of special approval where they reviewed my accounts with G and my websites, etc. From that info, I do believe they filtered the type of sites I could/would be rating. Not positive about that...
BTW - the interview/testing process is a pain. It takes about 10 hours of your time... but you do get paid for it.
They are very thorough.
|They don't want to listen to us kkobashi. I think you, Brett, and I are all on the same page. |
The problems with your argument are:
- We are not sure whether Marissa Mayer was referring to Google's 10.000 employees.
- Taking into account the poor quality of Google's serps in specific sectors until recently. And taking into account that 10.000 highly educated Google employees were doing the evaluation job. I wouldn't consider that as a good PR for the folks at the plex.
Better to balme it on the $10 raters ;-)
BillyS: Yeah and I'm sure all of those 10,000 outsourced India temps know how to read Chinese websites too :o)
I see the side of this where people may feel misjudged on why a site ranks low or high, banned or not. But I also see the side where there can be some justification of having borderline techniques for Ajax and Flash sites doing image replacement, flash replacement, essentially are simply hidden text but not made to stuff with keywords. Having some human check it out can see if the purpose is really a bad purpose or not.
They also hire humans to read blogs and forums to see what webmasters are doing and talking about. They work in Seattle.
|They also hire humans to read blogs and forums to see what webmasters are doing and talking about. They work in Seattle. |
They are watching us right now! ;-)
Being living in India and having several friends in Hydrabad working for big IT companies, I havent ever heard somebody working as an serach evaluator or whatever with Google... part time or full time.
Do they impose so harsh non disclosure policy?
There have to be many ex-evaluators who write blogs or post on Forums ;-)
"In order to storm Rohan we would need thousands."
"tens of thousands".
"but my lord, there is no such army..."
Cut to the news.
According to the ultimate Google authority (Wiki), they had 12,238 full time employees as of March 31, 2007 so virtually all of them would have to be pt/temp and if they are how could you count on them for QC?
Wasn't not doing hand manipulations some reason they put out in court when someone to dogde a lawsuit when somebody lost their rankings before? Maybe I've got that wrong...
|BTW - the interview/testing process is a pain. It takes about 10 hours of your time... but you do get paid for it. |
They are very thorough.
Sounds like they aren't stupid, either.
|Wasn't not doing hand manipulations some reason they put out in court when someone to dogde a lawsuit when somebody lost their rankings before? Maybe I've got that wrong... |
You're probably thinking of the SearchKing case (which SearchKing lost), but I don't believe Google has ever claimed that it doesn't do manual penalties. Indeed, the court decision reaffirmed Google's first-amendment right to adjust rankings or remove pages from its search results. Still, that isn't directly related to what we're talking about here, since--as we've just been told--Google's quality evaluators aren't doling out penalties or manipulating rankings.
This whole human-role-in-google-search thing is a farse, IMO.
Google has been preaching algo driven serps for years. Then few semi-human-edited and fully-human-edited-search-engines emerged.
Mighty GOOG got worried, and thought; lets jump on the human-edited-search-engines wagon.
And suddenly we see the kind Googlers busy telling the world; Ooooops...we forgot to tell ya that Google in fact has always been a human-edited-search-engine.
The remaining question to be answered by GOOG is: does the quality of human-edited-serps better than that of algo-driven-serps?
Good night and God bless.
10,000 silent OompaLoompa's perhaps?
Nobody ever goes in, nobody ever comes out... :-)
I know a woman who works as an independent contractor for Google, in search quality. About the only thing she'll say about the work is that she signed a non-disclosure agreement.
correct me if I'm wrong,
but 3 hors daily @ 15$ * 200 days * 10000 people = 90 million dollars per year - that's small change for a giant like Google, even if you add 50% on top of that for recruiting services and the like.
|This whole human-role-in-google-search thing is a farse, IMO. Google has been preaching algo driven serps for years. |
Search engines don't design themselves. Humans provide basic parameters and supply input, whether directly or via examples of relevant and irrelevant or good and bad pages for a "black box." Do a search for GoogleGuy's comments on how human evaluators are used, and re-read what maximillianos a few hours ago:
|"We were told our data would be used for research purposes and would not directly affect the search results." |
It's also worth noting that PageRank, which was one of Google's founding principles, requires human input. (Humans, not algorithms, create the links that serve as the "votes" on which PageRank is based.)
Is it me or is this suggestion absolutely ridiculous in the extreme!
To base your whole business model on algorithmic technology which was the whole reason people used you in the first place - lets be honest anything apart from search has been a huge, huge, huge (revenue) failure for Google. And then employ 10k individuals to "validate" and provide data to filter the algorithm.
What the hell are they doing then, the results have not improved - they have not removed spam, more legitimised it through mass ebay, amazon, wikipedia, shopping comparison, holding domains, mfa - etc. etc. We all know that if you are ebay then the rules are different than a small site that has missing in action for months or even years - I know that because I have been on the other side of it where I got to speak (on behalf of my rather large company in my capacity of technical director) to Google where they advised on how to correct indexing and ranking problems and within hours it was sorted (we are talking millions of pages).
What do these 10k people do - try various keyphrases to check the results? Try bloody acting on what webmasters in this forum have been talking about for years - 301s, URL hijacking, shallow crawling, incorrect duplicate penalties etc etc. And that is for free!
It is lucky I now treat Google with the contempt it deserves and now think absolutely nothing about it other than to make money out of their search results (that is my job as a webmaster) - I once gave a damn about ethics and "growth", "building", "quality", "longevity" and "trust", now it is so crazily volatile that I have to give a damn about me. I can't now call up Google to ask them to sort my indexing and ranking anymore as I am on my own now - just like everyone else who is not part of a huge site delivering millions of dollars to Google.
|lets be honest anything apart from search has been a huge, huge, huge (revenue) failure for Google. |
That sounds like a topic for the GOOG forum, not the Google Search News forum, but in any case, the statement is completely wrong, as one can see by reading any Google earnings report.
| This 106 message thread spans 4 pages: < < 106 ( 1  3 4 ) > > |