I wouldn't be surprised if there is another factor concerning relevancy. You can overdo the process.
Again, if you have a directory style link exchange going on, you are most likely a target for an algorithmic solution to an ongoing problem that plagues Google's indices. It's the easiest environment for an automated process to detect and since the level of abuse has reached critical mass, those that are doing it truly as a service to their visitors are going to be susceptible to collateral damage.
My personal opinion on this? If you are participating in link exchanges on a scale that is sure to send signals, expect those pages to become null at some point if they haven't already.
If you have all of your recips concentrated in one particular area, you may be asking for trouble. Don't shoot the messenger but if you look at those linking environments that are falling prey to the algo, I'd check to make sure that yours is not similar. If it is, there is a good chance your next on the list. :(
Anything auto generated with a footprint is also susceptible. In fact, that's right up there at the top of the list. Most of the abuse comes from users of automation. Once that automation reaches levels that Google is not comfortable with, there will be blanket issues to contend with. We've seen it happen in the past and history has repeated itself many times over.
Its time to find a better way to do things. I can tell you that a small mom and pop website that has an "unusual" number of links is probably going to end up as collateral damage. Those that are really rubbing it in the face of the SEs are at high risk. But hey, that's where they like to operate. ;)