Your suggestions were right on the mark!
There are many factors that affect how well a server can scale.
One factor I most commonly see left out in questions like this involves the content itself. Is it dynamic (asp/php/jsp/etc) or just raw html that this server is pumping out? If it is dynamic, how "well" is the code written? Scripts written without much regard to using as little resources as possible are not as scalable as cleanly designed pages intended to scale. Is there a database? That could be a limiting factor as well, in many ways. The database application itself may be just fine, but the way the data is queried may not be very effecient. On the other hand, you could have the best written queries (and indexes) in the world, but if the database application is not properly tuned, that can be the death of a server once real load hits.
If your site is just HTML, servers usually can scale quite well, usually way past what many people may expect. There are load testing tools available to see what the limits actually are. It gets more complex with dynamic pages, because really you are then testing an application, and not just a web server.
The real trick is to identify your bottlenecks, which is not always easy, then develop a plan from there accordingly.