Dave Naylor used to always argue (perhaps still does) that sitemaps are bad because they mask problems. His logic was that you can get crawled and indexed just based on your sitemap, but you'll never rank based on just that.
He would rather be able to see whether a page has the juice to get indexed based on internal and external links pointing to it. If you have no sitemap and a page gets indexed, this is proof that there is a link pointing to it that Google follows and that it at least has a chance of ranking.
This was back in 2005 or 2006 or so that he was pushing this idea (and not a peep since AFAIK) and now that Google has so many more sources of URI discovery, it seems like the diagnostic value of not having a sitemap is lost, but I always thought it an interesting counterpoint.