rish3 - 1:22 am on May 7, 2013 (gmt 0)
I don't think this accounts for all of the low-quality results, but one thing I've observed, in some specific niches:
a) Regardless of the "Webmaster Guidelines", there were niches that, for many years, were dominated by those that were manipulating the SERPS, often times 100% garbage sites.
b) The situation in a) enticed some webmasters that ran otherwise high quality sites, with great content, to also manipulate the SERPS. (kw stuffing, inorganic links, etc)
c) Over time, again for many niches, though not all...the SERPS started to get dominated by the people in b). They had not only great sites, good content, etc, but also had the shady backlinks. So, they effectively beat the total garbage sites with "match and exceed".
d) Google rolls out Panda, Penguin, etc. It punishes not only the total garbage from a), but also the "high quality" sites from b) that did some things they shouldn't have.
e) Whatever is left rises to the top. The "whatever is left" consists of
- sites with great natural backlinks, but old, or missing content
- mediocre sites with mediocre content
- garbage sites...but new ones, ones that were created after the last panda and/or penguin update and haven't yet been slapped.
- *maybe* some great sites, but only if the owner of said site was willing to maintain the site during the time period where everyone that was "cheating" kept them off the front page.