You might be doing better overall with all the link juice going to one "canonical" set of urls.
Duplicate content can divide your inbound link vote. Some sites may link to one url of a page, some to another. Eventually, Google is likely to drop one of urls, and you will have wasted a bunch of link juice.
With a canonical url problem, as you describe, you have a problem, sitewide, of not being able to control which version Google will display, and therefore not knowing which canonical to promote.