Google DOES NOT penalize for using Alt tags. For gosh sakes, they're there to improve usability! Google may not give them any weight, but they certainly don;t penailize for having them.
If you're a PR0, you've either been caught doing (or linking to) something bad (i.e. doorway page, spam, invisible text, etc.), or you've been caught by the mysterious "glitch" that snags any number of sites after each algo "tweak".