engine - 2:59 pm on Dec 16, 2011 (gmt 0)
Microsoft has released a paper titled "Domain Bias in Web Search." It's in a PDF format and you'll find it on the Stanford.edu site.
Here's the Abstract.
This paper uncovers a new phenomenon in web search that we call domain bias ó a userís propensity to believe that a page is more relevant just because it comes from a particular domain. We provide evidence of the existence of domain bias in click activity as well as in human judgments via a comprehensive collection of experiments. We begin by studying the diference between domains that a search engine surfaces and that users click. Surprisingly, we find that despite changes in the overall distribution of surfaced domains, there has not been a comparable shift in the distribution of clicked domains. Users seem to have learned the landscape of the internet and their click behavior has thus become more predictable over time. Next, we run a blind domain test, akin to a Pepsi/Coke taste test, to determine whether domains can shift a userís opinion of which page is more relevant. We find that domains can actually flip a userís preference about 25% of the time. Finally, we demonstrate the existence of systematic domain preferences, even after factoring out confounding issues such as position bias and relevance, two factors that have been used extensively in past work to explain user behavior. The existence of domain bias has numerous consequences including, for example, the importance of discounting click activity from reputable domains.
Here's the link [ilpubs.stanford.edu:8090...]