So don't use it then. Nobody's making you, not even Google. And if there are consequences (because there are always consequences, good, bad or indifferent) I'm sure you will accept them as what comes with your decision.
Authorship is a form of branding - it doesn't necessarily make your content better, or even more authoritative, but it does make it more recognizable. I know a bunch of people in my industry; there are even more people I don't know. If I'm looking for information on some topic, I for one would probably click first on the face I already know and trust. Authorship makes it easier to find. And in some cases, I think it also helps with the scraper issue (when used in conjunction with other things, like disabling feeds and using PUSH)