You can use htaccess to deny access to your site based on referer or IP.
This way, if you have spammy backlinks to your website you can block the visitors coming from the website that links back to you.
The question is if Google is also checking the referrer for that backlink or the crawler only builds a list of backlinks to your site and checks them separately. Because if it works by checking the referer you might be able to get rid of the bad backlinks.
How do you think Google is handling this ?