homepage Welcome to WebmasterWorld Guest from 54.226.168.96
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

This 106 message thread spans 4 pages: < < 106 ( 1 2 3 [4]     
Google's Human Evaluators - 10,000 of them?
tedster




msg:3390145
 3:06 am on Jul 10, 2007 (gmt 0)

We've known for a while that Google uses human evaluation in evolving their algorithm. They've even got a patent for integrating the humans and the algorithm [webmasterworld.com]. On 2007-06-23, Google held a Scalability Conference in Seattle. Here's an interesting tidbit from a Q&A session with Marissa Mayer:

Q: How do they tell if they have bad results?
A: ...they have 10,000 human evaluators who are always manually checking the relevance of various results.

Article by attendee Dare Obasanjo [25hoursaday.com]

Next time we're trying to figure out some odd change in the SERPs, we might do well do remember this human factor, eh?

 

pexcornel




msg:3392744
 5:01 pm on Jul 12, 2007 (gmt 0)

Oh Hobbs You wouldnt belive what I found in my logs from Google! :)
I'll just say this:
"We really appreciate your contribution to Ad Quality Check :-) "
:D

reseller




msg:3393075
 9:17 pm on Jul 12, 2007 (gmt 0)

Hobbs

Has the page you mentioned used to rank high for any specific popular kyewords?

Hobbs




msg:3393087
 9:32 pm on Jul 12, 2007 (gmt 0)

no, not at all, it's a trivial page as I said, that's why I was hoping to find the same IP visiting more pages on my site, but obviously they are evaluating one page not the whole site. Perhaps a whole site evaluation is algo based from the sum of single page evaluations by different busy bees, which is how the bot works really, no surprises there.

I also detected the same behavior in other sessions with the evaluation referrer.

reseller




msg:3393098
 9:40 pm on Jul 12, 2007 (gmt 0)

Hobbs

Those poor evaluators are paid only $10 - $15 /hr. That doesn't buy much search quality evaluation. Google gets what its paying for :-)

Hobbs




msg:3393103
 9:45 pm on Jul 12, 2007 (gmt 0)

or perhaps pay for exactly what they need ;-)

robertskelton




msg:3393112
 9:54 pm on Jul 12, 2007 (gmt 0)

Btw, did Google do anything to secure the objectivity (fairness) of the evaluators work? any special requirements?

There was a test, and that was it. I was recruited with a whole bunch of Google Answers Researchers, so I guess we had already proven ourselves through our work.

europeforvisitors




msg:3393113
 9:54 pm on Jul 12, 2007 (gmt 0)

Those poor evaluators are paid only $10 - $15 /hr. That doesn't buy much search quality evaluation. Google gets what its paying for :-)

How difficult can it be to tell #*$! from Shinola?

reseller




msg:3393122
 10:09 pm on Jul 12, 2007 (gmt 0)

EFV

How difficult can it be to tell #*$! from Shinola?

It could be rather difficult indeed. So much difficult that you need a whole bunch of 10.000 of those $10 - $15 evaluators to do the job ;-)

reseller




msg:3393134
 10:20 pm on Jul 12, 2007 (gmt 0)

robertskelton

Thanks again for feedback.

I read once that most of the evaluators/raters are young students, but that could be incorrect of course.

europeforvisitors




msg:3393141
 10:30 pm on Jul 12, 2007 (gmt 0)

I read once that most of the evaluators/raters are young students, but that could be incorrect of course.

$10-15 per hour for rating Web sites isn't a bad job if you're a student. It beats serving up grub in the university cafeteria or unpacking cartons in the bookstore.

Also, it isn't hard to tell #*$! from Shinola if you're intelligent and have been given guidelines. My 20-year-old son (who's a university student) would be an excellent candidate for the job.

Vimes




msg:3393347
 5:09 am on Jul 13, 2007 (gmt 0)

Hi,

Well 180 pages visited yesterday by a plex ip without the gbot user agent every single one of them from being supplemental to primary index and ranking where they were 3 months ago.

Hope it lasts

Vimes.

Gimp




msg:3393469
 9:31 am on Jul 13, 2007 (gmt 0)

There is a lot of talent available in Eastern Europe for 8 Euro per hour that does a lot of subcontract work for US firms.

It is not student talent.

They are very happy to get that money.

And there is a lot more that gets a lot less to do local work.

Google knows this and is opening a large operation to Eastern Europe to grab this talent.

In Eastern Europe you can get a lot more than you pay for. And Google is.

Fighting Falcon




msg:3396408
 12:41 am on Jul 17, 2007 (gmt 0)

Could this be one of them? Whilst reviewing logs I have often wondered who these guys might be. I have hundreds/ thousands visits from mountain view using various country specific google sites for the searches.

>>>>>>>>
from 72.14.192.9 in Mountain View, United States

Referred by www.google.co.uk searching for "ecommerce shopping cart"
14:48:37 Home page

Bewenched




msg:3400232
 6:14 pm on Jul 20, 2007 (gmt 0)

$10-15 per hour for rating Web sites

If they're outsourcing to india like everyone else then they aren't paid anywhere near that much.. not even close. Scary to think that someone in india is evaluating my pages. heck they can't even trouble shoot my dsl connection or get my dell order correct half the time. Just gives me shivers.

hippypink




msg:3401096
 12:24 am on Jul 22, 2007 (gmt 0)

clarification:

my friend who took the job lives in LA. $12 per hour, part time, with no benefits. Where I live, ave. non-skilled work averages $7 - $8 hourly, so this is a good deal.

The job was as someone mentioned earlier: is this page spam or not? and are trained to look for certain things.

If they are hiring overseas, then probably a diff. rate.

as we all know humans can add a different perspective to a that of a computers. this is smart way for google to test effectiveness of their algorithm in the real-world.

plasticstyles




msg:3416669
 11:35 am on Aug 8, 2007 (gmt 0)

I'm planning to apply for this evaluator job because I really needed the money, and nothing have come up so far on my job search due to lack of experience. If I don't a job soon, I'll be sleeping on the street. Anyway...

1. Is it easy to get this job?
2. What type of information should I put on my cover letter and resume to increase my chances of an interview. Samples would really help.
3. Do you need to provide references for this job?
4. Do you need to take some sort of test?
5. What type of people or personality are they looking for for this job?
6. I was a webmaster for my hobbies. Should I leave this info on or off my resume?
7. How many hours of work do they give you per week?
8. Does Google check your credit background for this job?

Ok, that's a lot of questions, but any informations you can provide would really help. Thanks.

This 106 message thread spans 4 pages: < < 106 ( 1 2 3 [4]
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved