ergophobe - 3:23 pm on Mar 16, 2013 (gmt 0)
No question that some sort of caching (or the combination of several sorts of caching) should usually be your first step if your server is bogging down.
This originally came up in the context originally of complaining (as I understood it) that a single page load was maxing him out and after a single page request he was having persistent high I/O and CPU activity and this was true even with plugins disabled.
That means that something is wrong on the server end, because Wordpress with very low traffic and no plugins should be fast without any caching at all.
So yes, in most cases people should look at the obvious. Especially for a content site with few logged in users, caching will give the biggest return for most people.