How do you know what's "dodgy" until you've already clicked it? ;) I'm no Internet newbie, but just recently I wound up on a "dodgy" site without any previous notion that there might be anything wrong - and I'm very careful when going around unfamiliar Internet territory. Granted, that doesn't happen often, but all it takes is once.
With Firefox, you can forget about most of those things - or, at the very least, you can forget the spyware checker. Also, Firefox includes a cookie manager, which allows you to delete only certain cookies. IE's only option is to delete ALL cookies, unless you do it manually in Windows Explorer, and I consider that pretty awkward.
But you're still essentially right - safe browsing habits will keep you pretty safe if you use IE. But I'd rather not take the chance - besides, Firefox has boosted my productivity so much that I couldn't go back now even if I wanted to!
On another note, I think IE on XP is quite a bit more secure than IE on previous versions of Windows. At least a couple of the security features you mentioned aren't available in my copy of IE (running W2K). I know XP is the norm these days, but there are still a lot of people running 98, 2000, and ME. None of them will have some of those essential security features, such as ActiveX installation warnings.