Yes, it could be. But a database could give better performance too.
Basically you are in one of three situations:
1. You're redesigning a site that already has lots of traffic and you need to be able to handle that traffic from day one. In that case, identify your peak load, try out a CMS with representative dummy data (many CMS have features or add-ons to do this for you for just this purpose) and then stress test it with something like Apache Bench or equivalent and see if you can handle the load and what sort of optimizations you'll need.
2. You're building a new site and you're worried about scale even though you don't actually even have traffic. In that case, I strongly recommend you look around the web for what Joel Spolsky and Guy Kawasaki have to say about building for scale before you need to (in brief: don't).
3: You're redesigning a site that's existed for years and gets 30,000 page views per month, spread out pretty evenly and traffic has been steady for two years. In that case, you don't really care one way or the other, because the average shared hosting account will serve this up without issue whether it's static HTML pages or a relatively slow CMS.
Personally, assuming you're not talking about a site with tons of real-time traffic like Twitter or Google Wave, I think you can get most of the advantages of the flexibility a DB provides with most of the speed advantages static files provide using a DB-driven site that caches static versions of pages or page components.
Then also remember that on-page factors might have a bigger effect anyway and serving up 15 javascript files and 12 CSS files and 32 background images and so forth for each page is going to create a lot of load no matter how the system is designed (see Yahoo's studies on the actual causes of delays in serving pages and where the big gains are to be made - look under YSlow).
As for the original question, think of if this way in order of increasing complexity
1. Each page is being designed separately and there's no dynamic component at all. In this case, there's not much need for a CMS at all
2. You want basic templating, but you're not really managing content via your site. In this case, you're just looking at some server-side includes in the language of your choice (or just using the Apache module for SSI). Clearly, flat files will win - no need to even create a DB connection.
3. You want something dynamic, but few complex relationships. So you want to grab an article and grab all comments related to that article, but without grabbing profile information for each author. In the fastest possible implentations, I think flat files would win. WebmasterWorld works a bit like this - each thread is just a file. I don't know the implementation details, about how it grabs your post counts and such, but AFAIK it does not, for example, create a DB record for each post in a thread, it just appends the post to the thread file.
4. You want more complex relationships. For example, let's say you want to show users who comment on a thread, all other commenters who have also commented on other threads that I have commented on. You could do this with tons of text processing, but it makes the head spin. It would likely be a fairly costly query to the DB too, but much faster to extract the data and it would be vastly simpler to design and maintain.
Also, a well-designed database (i.e. normlized) can handle queries that you've never thought of at design time without any change to the DB. The same is true for a flat file system of course, but if you're asking for something that isn't built in and foreseen, it might take hours to crunch through all the data.