From my experience working with sites of many different types, I think that xml sitemaps are a best practice for those sites with many hundreds of thousands of real URLs, especially if there is a lot of URL churn. But for even a couple of thousand pages, the xml Sitemap has never been my choice.
If a site of that size cannot get decent indexation with their inherent information architecture and link structure, then they need to know that - and upgrade those structures. A Sitemap can cover over the problem without really fixing it.
So for the first type of major size, high churn site I'd say the risks of removing a Sitemap are pretty high. For a couple of thousand pages, removing a Sitemap is probably not going to be a big deal.