It's been some time since I've done a thorough housekeeping on my sites. Waaaaaay to long if I'm honest about it.
So I've set time aside and started with a bad link check (used to use Xenu but now I'm an Apple man, I've used the Integrity app)
Sweet mother of Jxxxx! The link rot, especially with Government and quasi government sites, is incredible. Is there some unwritten law that says when there is a change of elected Government, or a realignment of Govt departments, that all websites have to be made over into something else or at least renamed. This is the one area I would have expected to remain relatively stable..... no way!
I'm seeing as much as 20% of all links returning a 404... which immediately raises a question.
If Penguin is about "unnatural link profiles" should we perhaps looking to see if everything is OK in our own back yard before racing off into disavowals, reinstatement requests and various other "technical" fixes?
It strikes me that when Google sees sites with such obvious signs of neglect, its probably not going to share the love.
Very few webmasters do this or at least very few of the people that come to me for help. We all think our website is perfect and Google is broken. If you are honest and regularly check your website you will find issues like this. The good news is that once you realize the specific issue it is much easier to fix.
It goes beyond a "good user experience". If the links can't by trusted why should the user trust the rest of your site?
Trust is a component of the "user experience" as it relates to how a user perceives the website and its contents. I'm surprised someone with as many posts as you does not know this already. Regardless, you can find a good explanation of what the user experience covers at Wikipedia or another site.