Which URL makes the best GoogleBot food?

All opinions accepted!

[msr986]

On one of my site redesigns, I am using ModRewrite to get rid of the ?'s in the URL's.

For Google, which of this two URL structures will make the best spider food for my keywords?

www.my-domain.com/product/page1/blue-widgets.html
www.my-domain.com/product/page2/blue-widgets.html

-or-

www.my-domain.com/product/blue-widgets/page1.html
www.my-domain.com/product/blue-widgets/page2.html

I like the second choice because it more accurately simulates a static html directory structure.

Which structure do you think will be better for the keyword search "blue widgets"?

Is one structure significantly better than the other?

-Marty

[mortalfrog]

I'd advocate the second over the first, but why not go the whole mile and do something like:

www.my-domain.com/widgets/blue-widgets/round-widgets.html
www.my-domain.com/widgets/blue-widgets/square-widgets.html

that way you get widgets in more often and cover more possible keyword cases. May as well replace every part of the URL with a keyword.

Of course, this might not work in your case - but my favourite goes like this:

www.blue-widgets.com/buy-widgets/blue-widgets/round-widgets.html

where you instantiate blue-widgets twice, and if you need more in your URL, you just use different key-phrases that look likely on the Overture word search.

Now paynt is going to come in and tell you that

[blue-widgets.mydomain.com...]

is better, but you probably shouldn't risk using canonicals unless you have really distinct categories on your site that are capable of standing alone IMHO.

[Marcia]

There was another thread that covered some of this recently. I think I'd be inclined to keep the directory structure as shallow as possible, and be careful about too many repetitions in the URL, just to stay on the safe side, where it looks "natural" and not too SEO-ish.

Unless you've got distinctly different products, regular directories should do it. Number 2 looks good, but maybe get a little creative and more specific instead of page1.html.

[Robert]

Unless you are only interested in Google, I've found to my great dismay the use of a hyphen can result in blue-wigits search meaning search for blue without wigits. If the site migrates to other seaches this could be a problem if people search for blue-wigets from memory not recalling the full link. This isn't a problem in the major searches, but it is in some of the secondary alternative engines and in some meta transfers of the search terms like dogpile sends out.

[msr986]

The purpose of this URL is to deliver dynamically generated pages, not to stuff keywords.

It uses the following parameters:

"Product" is a dummy directory. This allows the rewrite to recognize the URL. I could use another keyword here, but I have many different keywords that pertain to different products.

"Page#" is required by the CGI script to deliver the proper page number.

"Blue-widget" is required so the script can display the proper category of product.

I am not interested in spamming the Bot, but I might as well make the best use of my URL.

After much consideration I came up with the two examples. I am still interested in an option between the two (unless someone has a better concept!).

> I'd advocate the second over the first

Do you think that the second choice will do just as well as the first for the search term?

[msr986]

> get a little creative and more specific instead of page1.html

Any specific suggestions? The URL may return hundred's of "blue-widgets". These will be broken up into pages displaying 10 or 20 at a time. There may be dozens of "pages" returned. I though of using just the page number, but that really looks dumb!

> I've found to my great dismay the use of a hyphen

This sounds like another topic but, should I use a "_" instead?

[edited by: msr986 at 7:11 am (utc) on Aug. 5, 2002]

[Marcia]

I like the second better, no question about it. How will the site structure be, as far as what will be linked to from the index page to get to the /blue-widgets/ directory? Will the directory itself have an index page?

[msr986]

There is a static html index page that will display links to featured categories.

Near the top of the page, there is horizontal navigation which uses pull down menus. These links will cover all categories.

Both sets of links will call the same CGI script, which searches a flat ascii data file and returns the appropriate results.

The results are divided into multiple pages. Page 1 (of course) will be displayed first. There are links on each page to the previous and subsequent pages. These links will use the same format.

There is no "blue-widgets" directory, just a data file that will return "blue-widgets" results.

I finally get to put all my product in one database, and get rid of static html pages which are harder to update :)

Without the ModRewite, my URL's would look like this:

www.my-domain.com/search.cgi?keyword=blue-widget&page=1

[Chris_D]

Just a couple of constructive comments:

1. As long as you understand that every subdirectory 'takes away' one google pagerank 'point' - go ahead and have as many subdirectories as you like.....
i.e. if www.my-domain.com/index.html is PR=7
then www.my-domain.com/product/default.html will be PR=6
and www.my-domain.com/product/page1/blue-widgets.html will be PR=5

2. I'd use a '_' (underscore) instead of a '-' (dash) every time - unless you like % signs

If you minimise the subs, and use underscores - I think you'll get a better result.

This may also help:
[webmasterworld.com...]

[ferrari360]

hi Guys,

just a few clarifications..

>"As long as you understand that every subdirectory 'takes away' one google pagerank 'point' - go ahead and have as many subdirectories as you like..... "

I dont think this point is entirely accurate - the reason that it "appears" page rank decreases 1 PR per directory is because most websites have their "linking structure" set up this way..

www.my-domain.com/index.html (page A)
then www.my-domain.com/product/default.html (page B)
and www.my-domain.com/product/page1/blue-widgets.html (page C)

e.g Page A (PR 7) links to Page B (PR 6) links to Page C (PR 5)
the PR drops one PR "per link"

if you linked Page A directly to Page C you would find that Page C becomes PR 6.

- also a point with file naming if you only have 2 pages of widgets stuff you could name the first widget.html and the 2nd widgets.html (so you can capitalize on the plural form)

[ciml]

I think I'll be the lone voice in this thread. In my opinion, it does not matter.

Both /product/page1/blue-widgets.html and /product/blue-widgets/page1.html have "blue widgets" in the URL to the same extent, not that I believe that Google weights URLs much anyway.

[johnser]

I believe "_" is ranked higher than "-" in the ASCII table so a page called blue_widgets.htm will rank higher than an identical blue-widgets.htm
J

[mortalfrog]

perhaps, but you might want to check out this thread [webmasterworld.com] before using an underscore instead of a hyphen ;D

On keyword stuffing: I'm not advocating keyword stuffing - I wouldn't go and make a long URL just to have tons of keywords in it - but since you have to have a long URL you may as well put something relevant into it.

I mean, product1.html and "products" don't help the usability of your site, and they don't develop any more logical specificity, they're just there for your programming convenience. IMHO you should either rewrite your application so that you don't have to put so much into the URL (on my most recent site I do this by sending ALL requests to a my php application and then parse them and make the templates) or at least try and give that long URL some logical, structural significance so that the additional words help give the user a sense of where you are, and give the search engine more variety.

[mortalfrog]

also, if you look at the URL's here on webmasterworld, you'll see that even though the forums are in subdirectories they have the same PR as the root. I wonder why?