...but if your site is truly useful it will make a name for itself and your direct traffic and referal numbers will build steadily as well.
That is a GREAT long term strategy.
Unfortunately, you really need to gain some short term traction to be able to execute that long term strategy.
In my niche, I see some BRILLIANT sites that rank on pages 2, 3, 4, etc., of the SERPS. Their material is great (much better than mine), Yet I (and many other lesser sites) rank ahead of them.
Being so far down in the SERPs means that no one ever finds their great content (except for SERP sleuths like myself), and thus, no one ever links naturally to it. Hence, they never rise in the SERPs, and their great content continues to go undiscovered.
It's the search engine equivalent of a tree falling in the forest; if no one actually FINDS your brilliant content, does it actually exist?
BTW: There is a great link development opportunity in this if you are interested in lending a hand to those poorly ranking sites. It was pointed out to me by wheel several months back, so you know it has been tried and tested.