Magento - Pages appearing in supplemental index

[jaffstar]

We launched a Magento site, when searching for certain keywords / pages such as "brand Widget" , we have a page www.example.com/widget/brand.html and it's nowhere in the search results. After scrolling past page 4-5 I see :

In order to show you the most relevant results, we have omitted some entries very similar to the 467 already displayed.
If you like, you can repeat the search with the omitted results included.

When I select this , the page that should rank appears in the Serps...

Other pages on our site such as the homepage or www.example.com/red-widget/large seem to rank for this keyword but the page that is omitted.

What should I do to try and rectify this ?

[edited by: Robert_Charlton at 7:49 pm (utc) on Jul 16, 2016]
[edit reason] Changed to example.com... it can never be owned [/edit]

[Robert Charlton]

In order to show you the most relevant results, we have omitted some entries very similar to the 467 already displayed.

This usually suggests that your content is essentially dupe content or very close to what's displayed by everyone else selling the same thing. In the case of an ecommerce page not ranking, it suggests that you may well have used a generic manufacturer's description, and not offered any additional content or value that sets you apart from the rest.

From your post, though, it also appears that... in addition or instead of the above description issue... you are likely to have internal duplication issues, and these come from the way you've structured your site to sort different product attributes.

The urls you've posted suggest that you've set up separate pages in different categories whose urls include hierarchical paths for product attributes like color and size. I'm assuming that, for a given brand, the product descriptions are likely to be essentially the same...

www.example.com/widget/brand.html
www.example.com/red-widget/large

If I'm assuming correctly, you've got a situation where you're going to have a lot of internal duplication, since you've multiple urls to more or less identical content.
(I'm not certain that this is what you've done, though, as you're not including a page extension in your second url.)

Take a look at g1smd's posts in this thread, which is about dropping those hierarchical paths from the product page urls....

How important is it to organize pages into directories?
Sept 2011
https://www.webmasterworld.com/google/4364322.htm [webmasterworld.com]

We've had a lot of discussion about this over the years, but this was one of the earliest threads that touched on the topic.

[aristotle]

In order to show you the most relevant results, we have omitted some entries very similar to the 467 already displayed.
If you like, you can repeat the search with the omitted results included.

It seems to me that this happens a lot more often with Bing than with Google. That's my impression anyway.

I also think there's more to it than just duplicate or similar content. There may be a two-tier "index", and Google's upper tier may be considerablt larger than Bing's.

[aristotle]

Other pages on our site such as the homepage or www.example.com/red-widget/large seem to rank for this keyword but the page that is omitted.

This looks like a case of the "wrong page" getting the high ranking. I've seen this happen a few times on my own sites over the years. The page on the site that actually has the relevant information for the query is replaced in the rankings by another page, such as the home page, which has much less relevant information and isn't nearly as good a match for the query.

But usually this situation is only temporary, and eventually google gives the high ranking to the right page.

[seoskunk]

In magento set products to base url