I'm suffering from this filter/penalty/algo feature, and in other threads I've seen comments indicating I'm not alone, so I thought I'd start a dedicated thread.
The symptom: - You have a "red widgets" page, optimised for "red widgets". You naturally expect this to be the page to show up in Google for a search on "red widgets". - Google ranks pages with "red widgets" in the links to your "red widgets" page, not the page itself. All the work optimising your "red widgets" page, and "green widgets" is showing up instead, WAY down the SERPs :-(
I'd like to hear what people think causes this to happen, and if anyone has successfully recovered from this.
It happens a lot, and it can be a difficult thing to change. Google often seems to prefer showing pages further up the information hierarchy, possibly because they have more PageRank. So if higher level pages also have the keyword in anchor text, then they may rank above the actual "dedicated" page for that search term.
If this is happening for an important page, then channeling more link juice to the deep page sometimes does the trick. You can do this with internal linking, "PageRank sculpting", or even better, one solid backlink from an esternal site.