Having done both I can easily comment on this topic.
While it's easy to say you're losing so much money there's a pretty decent amount of infrastructure and knowledge involved to deal directly with advertisers. Most webmasters probably don't know what a media kit is, a flight, or many of the terms just to discuss the topic, let alone implement an ad rotation system, nor do they even have a merchant account to process ad payments.
Also, not all sites have the traffic volume to even warrant attracting advertisers on their own, nor are they staffed to handle advertisers, nor to they have the ad handling technology to manage the advertisers.
I happen to have the technology, the media kits, and self-service automated advertising signups to customers can spend as much or as little as they like buying ads 24/7.
Way back when I used to sell to really big companies (fortune 500) and had them running ads directly on my site sending me checks of $30K per campaign or doing $1 per click, etc. The problem was I quickly ended up working for big companies because every single day is was some new request about this or that, demographic, geoIP delivery, etc. It's more than a one-man show can deliver unless you no longer want to work on your site and do nothing but service the advertisers.
Eventually I let those big advertisers fall off my radar and switch to the self-serve smaller advertiser model, made less money but had a lot less headaches.
Then AdSense came alone and started giving me big fat checks for no extra work whatsoever.
Yes, I still sell directly to advertisers simply because I have the infrastructure in place and it doesn't take much effort to keep it going but I'm not sure if I started today that I would even attempt to sell direct again simply because it was a pain to set up.
However, if I were going to set up a direct ad system today to test the waters I'd probably use Paypal to sell ads and OpenX to deliver them.
Then of couse AdSense would be used to fill all the unsold inventory.
The simple truth is AdSense brings many advertisers and many publishers together in such an easy way for both the advertiser and publisher that it's really hard to break out of that model for either