Removing those keywords from your campaign should reduce your cost for impressions. While your CPC might remain the same this is still an improvement, because now your getting more impressions for the same amount of money.
If you remove keywords that have impressions but no clicks, then this will lower your CPC because you've lowered the amount of impressions required for a click, and Google likes this.
So to recap :)
Removing keywords that don't change impressions likely won't improve your CPC, but removing keywords that lower your impressions should improve your CPC.
Now, removing low CTR keywords (comparatively speaking) makes sense to improve your group's CTR. But don't just remove them. Move them to a group of their own and optimize those keywords with their own ads. This way, your whole account improves.
I don't think it will have an effect on CPC. But you could get more clicks, which is the whole point.
As a general matter of house-keeping I would suggest you remove keywords with zero impressions for the last 30 days from your account altogether. If you think they are pertinent and there are no impressions it's likely because you have the keyword serving elsewhere in your account that no impressions are showing.
Doing this helps to focus attention on the words that matter based on actual interaction rather than what you think they are going to do.
The only possible reason for not doing that might be seasonal campaigns, but I would suggest that those would have been offline anyway.
