I've recently started a blogging type site, it does depend on what blogging software you are using.
First off I would say make sure you have some software that is able to 'ping' blogging aggregator services like technorati, weblogs.com etc, theres lots of them out there, this is also a feature of feedburner (more on that below).
If your blog has an RSS feed, which it should have, put it in feedburner, this will give you a lot more options to do with the feed and let you track readers, it will also publicise your blog links and will allow them to 'ping' blogging sites for you too.
Make sure that the site is not too complicated and does not have too much in the way of 'asthetics' personally I like to be quite minimal in all of my design work, but a blog site let me go as minimal as possible, make sure the bulk of your HTML is actually the content. Content is king with blogs, not to mention make it unique, even if you just change the colour scheme on the default blog/cms software I hate seeing 1m blogs all looking exactly the same.
If you think you've posted a particulary good article add it to stumbleupon, digg and del.icio.us. This not only gives you a few more backlinks every month, but it gets your site out there.
Post good and unique content (as always), this is the most important part of the blog (I should hope this part is obvious).
If you follow the above few steps you should see organic links popping up, I get links from allsorts of websites including some high authority sites. Then its just a case of content, content, content.