n00b1 - 6:37 am on Jun 14, 2013 (gmt 0)
It isn't that simple. It is natural user behaviour in some circumstances to want information from multiple sources. If they are reading reviews, for example, they will likely want more than one opinion. There are many instances where clicking back to the results page is perfectly natural and not at all an indicator that the user didn't find the first result useful.
John Mueller has said time and time again that that sort of signal isn't used by Google. I know it's a hard thing for many here to take in who are absolutely obsessed with user metrics, but it's just how it is.