enigma1 - 6:10 pm on Aug 15, 2010 (gmt 0)
I meant that if you are trying to preserve that backlink's equity then you should have content related to the original content for that URL
Yes, but I think if the URLs were removed for years and that's from what I understand is the issue, the spiders insist of accessing them today and presumably for years to come. And IMO this usually happens, because the URLs are posted outside the domain. I see it on my site too btw.
The best I have come up with is to dynamically try and locate the closest URL that matches the request and redirect there with a 301 instead of using the root. And that doesn't mean the original request is always relevant with the redirect but it may work for some cases.
If I was having a 404 that wouldn't change anything in the spider's index. The next time it will crawl the same external site's page and come in again getting another 404.
The result of all this is the spider wastes b/w from me on the one hand and on the other, wastes its resources instead, it could crawl a valid page.