martinibuster, are you talking about images that show up on page load, or clickable links? A clicked link may not be so bad, because there's a chance that the surrounding text says something about your site-- and the user's address bar does at least end up showing your sitename. But there's no way for your server to distinguish between this and a "pure" hotlink; all it sees is a referer.
If you're talking about images that show up on page load, there's another aspect. Hotlinking creates a horrendous experience for the user. You don't notice it if a page just has one or two hotlinked images. But when a page has dozens of hotlinks, the thing takes hours to load up. Sometimes it's because the site owner is an idiot and has vast images resized in html. But it's also the time spent looking up all those separate hosts. The nanoseconds add up.
Well, this is a very interesting discussion thanks to your contributions! :)
Lucy's right about a page with hotlinked images loading slower. This is an issue that's particular to forum pages where members hotlink to image hosting sites. Although there may be value in hotlinking out to images, I would like to focus the discussion on our own images being hotlinked.
For the link building value, I really like the idea of watermarking a hotlinked image with the originating URL. Anything to bring awareness, mindshare, and traffic is good.