Despite what Google or any other search engine thinks, I even link to sites which have been deemed "bad neighbourhoods".
This could be part of your problem. Google has been warning for a long time to not link to bad neighborhoods.
I recently read a discussion by g1smd on WW where he tested linking to a bad neighborhood from a frequently cached page--and noticed Google quit updating the cache the very next day. He removed the link 2 weeks later. He did this 3 times with the same result.