I guess the first question that has to be asked is: What kind of search traffic do those 5000 pages attract? Even if only a couple of referrals on average I might be reluctant to take them down.
Then, would the 20 core pages stand on their own to attract referrals? Are they somehow supported by the 5K?
Drastic step, big decision.