tedster - 4:22 pm on Jun 1, 2011 (gmt 0)
Whether to 301 or 404/410 should be a decision that rests on actual data, and nothing else. Here are the two data-driven questions about the URL being deleted:
1. Does it have a decent backlink profile - something worth preserving?
2. Does it get ENTRY traffic?
If a URL is one of 97,000 being considered for removal, I'd expect less than 1% will get a "yes" to one of those two questions. In a similar situation on a recent project, our team found 80 URLs worth redirecting out of 30,000.
I prefer a 301 redirect that goes to a well chosen URL, one that matches the value and function of the removed page. I do not use a broad brush approach that redirects them all to the home page. I even avoid a high level category page as much as possible. 301 says "this content has permanently moved to a new location" and I take that rather strictly.
A 404/410 status is much quicker for a search engine to process. Processing 97,000 URLs that use a 301 redirect requires a whole lot of trust checking because 301 has historically been a major spamming tool.
My guiding rule is to redirect as little as possible. I've seen many sites that have generated their own ranking troubles over time by being very casual with the 301 status.