Robert_Charlton - 1:54 am on Jun 9, 2010 (gmt 0)
The three most disturbing things about the last 2-4 months for me
Thinking out loud here, and I'm not a search engineer... we can either attribute these three things to the new algo, or to the transition to Caffeine, or to a combination of these. Presumably, the new algo anticipated some of Caffeine's capabilities and possible vulnerabilities, but you never know. ;)
I can see where the slow and random indexing might be due to a limitation of resources as Google migrated from one file system and infrastructure to another. Conceivably, old data had to be cached until new data migrated to the new system. This might also explain lack of direct results. If this is the case, with the announcement of Caffeine we should be seeing a more complete and fresher index soon.
The proliferation of dupes and mulitiple site spam might be due to cached data coexisting with new data... or it might be a vulnerability of a system and/or an algo that incorporates super-fresh data. Presumably, someone anticipated this as well, but perhaps not.
I doubt that any of these behaviors was what Google desired, but they may have been what Google felt they could live with during the transition. I'm not talking about the changes that are clearly algo changes... just about these last three particular things that scottsonline has noted.