I remember reading a long time ago that pt was really intended for print media, and there were many good reasons to not use it on a website. I always thought em was really the way to go, so that the text size could change along with the rest of the page, which should be liquid so it conforms to the viewers settings.
I've never had much luck with em, as my site has a lot of graphics and images, and I really need things to be situated as designed on the page. So, I use px for consistency.
I just checked, and changing the text size in IE when viewing my site doesn't increase the text size, but the zoom works and that increases image size as well as text, so anyone who needs bigger text would likely also need larger images.
I've never had anyone complain. It would be interesting to hear what the latest thoughts are on this, as I'm sure using px is not the best way to go. I would like for my site to be flexible for the end user, but my site would just be confusing with large text that wraps around images that remain the original size and are no longer aligned correctly with the text.