IMO Google's continual changing of the way that search results are derived is driven by three fundamental issues:
1) The need to increase profits, maintain their share price and keep Wall St happy. As a publicly listed company, they have no choice in this.... how they do it is a whole other topic.
2) The need to keep ahead of the "bad guys" that see Google as a money pit and will use any trick that works to their advantage. Google will do whatever it takes, and collateral damage be damned, to prevent this and regular SERP upheaval is a way to do that. The skewing of results to favour big brands was another way of locking out spam sites.
3) The Google of today only pays lip service to search intergrity. They are an online advertising platform and have been for some time. The real value in SERP's is to provide screen real estate that carries Google's products and services. Organic results have been made largely irrelevant as they are pushed further and further from the public view.