I'm going to make a gross assumption here, but please play along.
Between the blog network de-indexation and the Penguin update it's hard to deny that at least there is a real possibility that links can harm a site. That's the assumption. These could be from Negative SEO, bad prior habits, or any brand of potential harmful scheme. The source doesn't matter for this debate.
The question I'd like to open is: Should webmasters spend time working to remove bad links? Should they be aware of "unauthorized" links and actively work to remove them? Should they police their own link profile to prevent overuse of keyword anchors? If they should, what is the time split.