pontifex - 1:21 pm on Nov 3, 2010 (gmt 0)
re "conspiracy theory":
here is my take...
1. G does know very well what they are doing
2. that does only apply to a core team of around 10-15 people
3. G has had and still has the goal to achieve the maximum with as little effort as possible (coding, hardware, etc.)
4. Calculating a relevancy set of URLs out of billions of pages with 100+ factors is just Mathematics, not rocket science!
5. 10 people tweaking a formula for the rank of billions of URLs and then 10 more people trying to combat SPAM with more formulas after that (I still believe more than ever: there are many calculation rounds - first rank, then filter, iterate!) creates the ocean of the SERPs. And that is in constant change...
As I understand companies, I really do think we are in the hands of around 20 core people behind the core product of Google. The rest are maps, mail, admin, sales, network engineers (poor guys just plug boxes into racks and configure routes) and so on.
There is no uncontrollable anonymous Google, they are just damn good to keep these 20 people out of the line!
So: there are only 15-20 geeks tweaking a set of formulas... each with an organizing mind on top of the these 2 core groups. That's all, the rest are urban legends and a bunch of underpaid guys driving through the world with cameras on top of their cars!