I have once installed such a system in a corporate environment but it was based on the closed source programs MS Exchange and Mailmarshal. It did pretty much everything you mentioned here. But such a solution has a price tag.
Most open source email systems focus on functionality, rather than restriction, data security and administration. Of the existing open source projects, I think that the Courier Mail server comes closest to your needs. It has a versatile IMAP connector which allows connections with Squirrelmail, SOGo and other user interfaces, user quota can be configured and it has some sort of message filtering.