No they should not be expected to know that. But, they also shouldn't penalise an entire site because they have sent people for certain key phrases that provided a negative experience.
True, but we don't know for sure that they're doing that; it can only be speculation. And in fact, in most of the Panda cases that have been shown to me, it's not the entire site that's been brought down.
It would be interesting to know if, on URLs that have been severely affected by Panda, they are suddenly ranking for long tail, not-well-matched keywords (that they WEREN'T ranking for pre-Panda). I'd expect that information could probably be found in Analytics/GWT.