EditorialGuy - 5:07 pm on Jul 11, 2013 (gmt 0)
I still see rows and rows of Googlers sitting in front of machines running queries and grading what is returned.
That could explain why some searches yield a mixture of pages from big-name sites (Wikipedia, Amazon, TripAdvisor, etc.) and an almost random sampling of smaller sites on page 1. The biggies get evaluated (and fall into the testers' comfort zones), while only a minority of smaller sites (even the good ones) are lucky enough to get plucked from the virtual hat for grading. (Mind you, this wouldn't explain why blatant spam sites rank high for some searches.)
Still, let's be realistic: The "rows and rows of Googlers sitting in front of machines running queries and grading what is returned" approach isn't scalable. We know that, and--more important--Google knows that.
Google has always said that human evaluators are used for QC checks of the algorithm, not for deciding or directly influencing what ranks and what doesn't. That assertion is a lot more reasonable (and, to me, a lot more convincing) than the notion that Google's data centers are filled with little elves whose collective judgment determines whether an Amazon page outranks John Doe's page in a search on "buy green widgets."