Rewriting dynamic URLs and avoiding duplicates

[lhtrading]

On our Coldfusion e-commerce site, we sell lots of different kinds of widgets which are in categories. Each Category is assigned to a Group.

A widget might be in more than one category - a small blue widget would be in Blue Widgets and Small Widgets.

The categories and groups are named terms that people search for.

At the moment, our pages look like this:

/store.cfm?group=123
/store.cfm?cat=678
/product.cfm?p=5432

Which is a bit old skool. ;-)

Are we right in thinking that it's better for ranking purposes to have our pages use a static URL structure like this:

/widgetgroup/
/widgetgroup/bluewidgets/
/widgetgroup/smallwidgets/

But because a Widget can be in more than 1 category, we'd get a duplicate page:

/widgetgroup/bluewidgetcategory/widget1name.cfm

would be the same product page as:

/widgetgroup/smallwidgetcategory/widget1name.cfm

And that's bad.

How have people dealt will this issue before?

Thanks!
Richard

[g1smd]

Dynamic URLs can be indexed. Dynamic URLs will rank.

The problem comes when there are more than three parameters.

Even with two parameters, another problem comes if one link has ?a=1&b=2 in it, and another has ?b=2&a=1 in it. That is classic "duplicate content".

Those are the factors that hinder ranking and indexing, not whether the URLs look like folder-based URLs.

Your design with the same page "appearing" to be located in multiple folders is a big problem. It should appear in only one of them. If you can't manage to do that, then all of the others should have a "meta robots noindex" tag added to keep them out of the index at the very least.

[lhtrading]

Many thanks g1smd - appreciate your reply (and your posts in *so* many other threads).

Will certainly ensure that there are no duplicate pages as described.

Is there an advantage in using a folder structure because keywords can appear in the folder names?

Or because the tree of folders can give context for what the end page is about:

For example:

widgettype/widgetbrand/widgetname.cfm

tells Google that the page widgetname.cfm has something to do with "widgettype" and "widgetname" in a way that product.cfm?A=1&B=2 does not?

[frances]

I had exactly the same potential duplicate content issue very recently. What I did was use rel="nofollow" for the possible duplicate links. I dont know if this was a good idea or not but google at least seems not to be indexing the duplicates.

Does anyone know if using robots exclusions - on the page or in robots.txt - would be a better solution?

[g1smd]

The nofollow attribute tells Google that you don't "trust" that link and that you don't want to pass and PageRank to that page.

It doesn't stop it being indexed if someone else still links to it.

I would instead use the noindex tag on the page that you don't want to be indexed.

[Bewenched]

try doing each link like this

/widgetgroup/smallwidgetcategory/widget1name_PRODUCTCODE.cfm

this will eliminate the overlap

or like this

/widgetgroup/smallwidgetcategory/productcode.cfm

sometimes the names get to looking spammy

[g1smd]

Not quite. Do NOT use underscores in the URL.

Separate any words using hyphens or dots.

Avoid using spaces or undersores, always.

[frances]

Thanks g1smd. I will use robots instead.

But why not use underscores?

[Pirates]

Not quite. Do NOT use underscores in the URL.
Separate any words using hyphens or dots.

Avoid using spaces or undersores, always.

Good advice but if you have pages already ranking using this format you should retain and think of this as future content advice.

[g1smd]

A search for "two words" will find "two words" and "two.words" and "two-words" but will fail to find "two_words".

[Pirates]

Nah sorry I think the advice on old urls is wrong its far better to maintain them as its aged content whatever the url. On future content I would agree that the format you suggest would be better. But on aged content it would be suicide to change url names.

[edited by: Pirates at 2:27 am (utc) on Nov. 14, 2006]