SteveWh - 11:16 am on Mar 15, 2012 (gmt 0)
I can confirm your results. With the AddHandler line in effect, I get a page, but when I comment it out, I get 404. I did the test in Windows with Apache 2.2 with PHP running as an Apache module.
The only insight I can try to offer is that Apache might be using a different subroutine to figure out what to serve (if anything) for the requested URI, depending on whether it's going to serve the page itself immediately or run it through PHP first. Maybe there are two different URI resolution and file-find routines.
Your discovery is something I haven't seen discussed previously.
This could be a minor security issue, since it provides a way for an outsider to determine if .htm files are being passed through the PHP interpreter, and thus whether .htm/.html pages likely contain PHP code upon which PHP exploits could be attempted. (On the other hand, it's more efficient to just send the exploit requests without bothering to test for underlying PHP first.)
Offhand, I can't think of a good reason to create these two different behaviors on purpose, but it's always possible there is one.