TheMadScientist - 5:10 am on Jun 9, 2010 (gmt 0) [edited by: TheMadScientist at 5:12 am (utc) on Jun 9, 2010]
I understand your point a bit better, but IMO they are flooding an 'old' algo with information and sectioning the results, so before (Big Daddy, GFS I+) there were parts of the index updated in 'near real time', but now they have the ability to update everything in 'near real time' and it's not what the algo has been doing, so they significantly increased the information available to the algo and started running it way faster, along with a change in the way long-tail searches are ranked, so what happens?
Everything looks all 'out of whack' because of the speed and information increase (or the information for a particular sub-set of results possibly decreases, depending on the sectioning of information), which means the algo needs to be adjusted to work with the new storage system more reliably.
I think we're seeing as much 'floating spam' as we are because they haven't adjusted the algo fully to Caffeine and that's what I would expect them to be doing over the next weeks and months, so I think it's basically the opposite of what you're thinking and the issue is the algo has not been fully adapted to the Caffeine 'way of things' yet.
BTW: You're right it's been a fun discussion. Peace. :)
[edited by: TheMadScientist at 5:12 am (utc) on Jun 9, 2010]