Never had a problem on previous windows servers so I assumed this was working after I changed from 2003 to 2012, but today I needed a rather more complicated redirect and found the custom 404.asp I've always relied on is simply not being run at all.
I've set (and checked!) in errors for any file-not-found to go to /404.asp which in turn does a few basic checks before admitting defeat and issuing a 404; checks such as is the extension requested .html from an old site link instead of the newer .asp, for example.
I suspect there is something that needs setting in addition to the 404 error handler but find it I cannot. :(
The code still works fine on an old server.