tedster - 6:32 pm on Aug 15, 2010 (gmt 0)
wastes its resources instead, it could crawl a valid page
That does seem like a reasonable assumption - but I haven't seen the crawl budget work that way. And a 404 doesn't use up much bandwidth anyway.
The best I have come up with is to dynamically try and locate the closest URL that matches the request and redirect there with a 301
That sounds good to me.