The sitemap is always an optional extra. If everything on your site is linked from somewhere else on the site-- or somewhere on someone else's site-- google will find it anyway. Sitemaps are only necessary if you have a section of your site that isn't linked from anywhere, but you still want search engines to index it. (Query: Under what circumstances would this happen?)
Conversely: Once you've got something on a sitemap, google will remember it forever. Doesn't matter if the page disappeared in 2006 and you've created eighteen new sitemaps since then. Google will still crawl it periodically, and will list it as a Crawl Error and claim that it's On Sitemap.