Jez123 - 9:49 am on Oct 11, 2012 (gmt 0)
Google still has a record of the keywords you used in the anchor text of any purchased or artificially-built backlinks. So even if you remove the backlinks, the site could still be penalized for those keywords.
If that is the case then there is no defence except to forget and move on or sink. Unless the penalty is lifted after time of course.