I use a template for the stuff that appears on every page of my site. I use the "library" for stuff that appears on several pages. For example, My main page is about Widgets and I've got 3 pages of details for the widgets.
X (a color of widget pulled from a database such as RED widgets or WHITE Widgets) are the types of widgets. Now, I've got "X Widget Main Details" "X Widget Specs" and "X Widget Related Reading/Links". I want to easily navigate between those three pages (while still staying with the same "X" variable) but I DON'T want that navigation bar to appear on every page, just the X Widget detail pages. So I create a library and slap it into those three files. That way, if I decide that I want to add "X Widget Pictures" I can build the page, slap the library item in that one page, alter the library item with the new link, and upload four pages without having to manually open those three other pages to update the nav menu.
Confusing? Yup, but that's the widget business for ya. ;)
Somewhat similar to Grumpus,
I use templates and the library for site assets. For example. currently building a site, which has several templates, denoted to sections of the site and theme's, dependant of the sub-depth positioning. The library items, i use to make the page even more particular. Hence by using enough templates, then by using library items, you can pretty much, build the structure of a big static site, where every page has considerable difference's if you have taken the library facilities into account before designing the templates.
I'm currently re-working a site, as it has not been altered since this time last year, hence the original structural mistakes, have only been covered up by small sectional updates, rather than a site wide global fix.
I use library items for things like copyright footers, that end up on every page. You update the item, it updates it sitewide (if you want it to). Pretty cool.