Yes, Google sometimes mis-matches a page to what the searcher really wanted. In such cases the user metrics will tend to show that it is a mis-match. So The Google algorithm will learn that it's probably a mis-match and drop the ranking of the page for that particular search term.
On the other hand, sometimes the page will be a good match for what the searcher wanted, and the user metrics will so indicate. In this case, the algorithm will improve the ranking of the page for that term.
So over time, as more user data is collected, the number of mis-matches will be reduced and the SERPs should improve.