diberry - 5:33 pm on Jul 12, 2013 (gmt 0)
Besnette, that's really interesting. I've been trying to put it all into perspective lately, and this is the best "timeline" I can come up with:
In the early 2000s, Google's working great. In about 2003 Google introduces Adwords/Adsense. Spamming ensues. Google spends the next few years fighting the spam with mixed results, but mostly the SERPs are still pretty good. But at the same time, big brands and stores are realizing they were wrong to neglect the internet, so they start hiring SEOs. There's a thin line between spammers and SEOs. Plus, online content is growing rapidly out of control. Google knows they're going to have to move to something AI related - the algo engineers can't keep up with all this forever. Maybe Panda was the beginning of machine learning - maybe the algo does the first pruning, and then Panda kicks in so it's dealing with a limited subset (not so overwhelming).
But the bottom line is that it simply isn't working. Your ex-engineer may be right - maybe in the end, better sites will prevail, if they survive. But right now, it's a mess because:
--AI/machine learning is new and just not up to it yet
--When Google fights SEO, they are fighting brands, and the brands are willing to spend ANYTHING to beat them. Or maybe they've just realized this and thrown in the towel to reduce the job of the algos/AI/engineers to something manageable.
In short, I just don't think Google can keep up anymore. I think even with enormous resources, the purest of intentions and the smartest people in the world, it is just not possible to index the entire freakin' web in a meaningful way, and that's Google's business model for search. (Bing seems to act more like a curator, which may be the more sustainable model in the long run). Google may yet get this worked out and come up with a way of indexing the whole web in a way that's satisfying to most visitors. But right now, we're a long way from that.