For certain queries that seem navigational in nature, particularly when looking at brand names and peoples names, it seems like Google intentionally returns results that are diverse in sentiment.
While monitoring the top 10 for a client, I continually see sites appear on page one that really shouldn't be there. Whether it's a forum post with the name in the forum or a negative listing with 1 or 2 links, these sites consistently outrank pages on older, more authoritative domains.
I'm continually dumbfounded by this and was wondering if anybody would be able to give some ideas as to why this may happen.
The answer is yes. Beginning early this year Google made an increased effort to expand diversity on the first page. Several of the top people made public comments about it.
There's been some discussion about how Google flags certain queries as "deserving diversity" just as it flags other queries as "deserving freshness". The specific details must involve their huge pile of user data, I'd say.
The "diversity" issue also bumps into "disambiguation" and "query expansion". Those are pretty near concepts, although diversity seems to be a bit wider. For instance, diversity might include giving a special break to a decent website that so far hasn't been able to rank well - the mom and pop type with no SEO knowledge.
Thanks for the response. This is good to know for reputation management issues.
If I'm creating content to rank for "Example.com INC", I'd ideally have articles/sites that are about history of john doe, locations of john doe incorporated, etc., with different link sources to each so Google would deem it as diverse.
One thing I've noticed when trying to occupy many spots in a given top 10 is that regardless of all the linking and content creation that is done, there are still 2 or 3 sites that appear in the SERPS, which are very non-competitive, that we didn't create and can't displace. Perhaps it's because the link sources are similar so Google intentionally bumps up some other pages that have different link sources as they are more 'diverse'.
I also work on reputation management at times - but the good thing about diversity in the search results is that they are simply not relevant for the client. Neither negative nor positive, and therefore not a reputation issue.
In some cases I've considered helping a "diverse" url to rank better - it sure beats seeing some wacko comment on a blog.