Faceted Navigation Problems and Noindex

[Planet13]

Hi there, Everyone:

I wonder how the best way to implement faceted navigation is on an ecommerce site (and I apologize if this is not technically "faceted" navigation).

We sell widgets, and the most highly ranked (and most visited) pages from the search results are category pages for the widgets separated by brand. For Example:

sitename.com/brand-x-widgets.html (a category page)
sitename.com/brand-y-widgets.html (another category page)
sitename.com/brand-z-widgets.html (another category page)

However, these widgets have various other "attributes" and have different pricing points as well.

So we would like to have "categories" based on price levels like:

sitename.com/widgets-under-10-dollars.html
sitename.com/widgets-under-20-dollars.html
sitename.com/widgets-under-30-dollars.html

Since those price point categories have little chance of ranking well at all, what is the best way to deal with them? The products that will be listed on the price point categories will be the SAME products that will be listed on the brand pages, and I want to avoid canonical issues.

I think it would be best to noindex those "supplemental" categories. What are your thoughts?

BTW: The final product URLs will NOT have the category names in the URL, so it doesn't matter which path (i.e., through the brand pages, or through the price point pages) the visitor / bot takes to reach the product page. The url will always be:

sitename.com/widget-product-code.html

[Robert Charlton]

Using "noindex,follow" in your robots meta tag would keep the pages out of the index and would also recirculate PageRank going to these pages throughout the rest of your site.

Definitely avoid rel="nofollow" or robots.txt to control indexing, as either of these would block spidering and create a loss of PR. Also make sure the faceted pages have good navigation back to your category pages.

I think it's important when doing this kind of faceted sorting to keep it low in your nav structure where you can, to minimize the PageRank drain from important parts of your hierachical structure. While you will be using the "follow" attribute in addition to "noindex", I don't know that the price sorting process is link juice friendly.

You're being smart to make sure that your final product urls remain consistent. If you use absolute urls in your code, you could include folders for categorization if you wanted to... but you'd have to be careful not to change these if the product is linked to from different pages in your site.

[Planet13]

Thank You, Robert:

I think it's important when doing this kind of faceted sorting to keep it low in your nav structure where you can, to minimize the PageRank drain from important parts of your hierachical structure.

That is going to be a bit of a challenge...

I know that USERS will expect the links to those supplemental sorting pages to be up front and on the table, as it were. Many of the large sites will put links to those supplemental category pages at the TOP of the category tree...

This is definitely going to require some careful planning.

[tedster]

I've found that Robert's guidelines sometimes need to be modified if the scale of the site is very large. For example, I worked with a site that really did need to use robots.txt to stop the crawling of all their "sort" URLs.

They had over 12 billion potential URLs when the various sorts were taken into account, and googlebot spent so much time spinning around in the various nsorts of essentially the same content that too many important pages were not getting crawled.

[aakk9999]

There is another possible way to deal with this and this is to consider using POST rather than GET for unwanted faceted/sort URLs.

You could then have your "under $10", "under $20" as your POST parameter and post all these options always to the same URL. If you are on IIS then "postback with options" is perfect for this.

This of course means that any external linking to such page would only show whatever the page looks like when the POST array is empty.

[tedster]

I've also seen ajax used for sorting - though I have only been involved in one such project and it was small scale.

If you don't want ajax URLs indexed, you can use the hash [#] for the sort parameters, which Google will not treat as a "new" URL. And if you do want the ajax sort URLs indexed, then you can use hash bang [#!], which Google will index as a separate URL. Making AJAX Applications Crawlable [code.google.com]

[Planet13]

Some interesting ideas and suggestions.

It will not be a huge site (only about 120 products or so) but I have very little external page rank flowing in, so I really don't want to squander it.

[Planet13]

I am not really sure it is technically a sorting issue. And if it is ok, I will use a more or less real world example, if that is ok.

The site will be selling clothing. There are a few particular brands of dresses (Brand X Dresses, Brand Y Dresses, etc.,) that we sell that have low search volume, but even lower competition, so we rank pretty well for them.

As you know, there are many ways to classify dresses (Long dresses, V Neck Dresses, Summer Dresses, etc). The competition for those keywords is too high for us to rank for them. However, we would like to have those as categories to help user navigation.

So most of our search engine traffic that comes looking for dresses comes in through those Dress Brand Pages, but once they are on the site, then I want them to be able to easily find the other styles (V Neck, Long, Mini, etc.,) easily - all without messing up my rankings.

[g1smd]

Having ONE URL for the product is the major part of the battle, glad to see you recognised that fact early on.

That's the point that is most often overlooked.

From the product pages, you can link back to multiple category pages: "find more: this, that, other" and so on.