URLs: To "/" or NOT to "/". That is the question

[DansSitesRScrewed]

There seems to be some mixed feelings about this, but I wanted to get everyone's opinion.
We have a dynamic site and in the category pages there is an option to have the suffix in several variations. Really it will allow you to add any suffix you want. But my question is which is the best way for SEO purposes, etc.

So the options are www.mysite.com/category (without slash) or www.mysite.com/category/ (with slash) and of course the www.mysite.com/category.html

Now I have definitely read many articles that stat it is a bad idea to end a category url in the .html, so I feel confident on that one, but the version with or without the slash is the one I would like clarification on. I have read a few articles that tend to lean towards the url with the slash at the end. And I guess actually the home page as well would be included in the mix?


Thanks
Daniel

[JD_Toims]

Doesn't make a bit of difference in rankings.

I'm not even sure why someone would try say .htm or .html is "bad" as far as rankings are concerned, because they're not.

Check out some of the rankings -- Do people really think here or anywhere else would rank better without the .ext in the URL? Ugh! at the FUD they're spreading if they do.

PHP.net uses .php extensions.
w3.org uses .html extension.
wikipedia uses extensionless.
eBay uses .html extensions.
walmart uses .do extensions.
Apache.org uses .html extensions.

I could, but will refrain from continuing.

[DansSitesRScrewed]

Good point. But then again you are talking about some big boys. The main reason I ask is because Google seems to be having issues with our category pages. So I was ruling out ANYTHING that might be the cause of it. And that question was on my list. Seems Google is considering irrelevant pages for our category keyword terms. Bing has it right. So could be something like over optimization, but trying to rule other things out.

[JD_Toims]

I can't list other, smaller sites that use/don't-use extensions here -- The results tell the story and .ext, trailing /, or extensionless makes no difference at all in rankings.

Does someone really think one page is going to outrank another page for a query because one webmaster decided to go extensionless while the other left .html on the URL? WOW! if they think things are that simple to manipulate in search, circa [essentially] 2014 -- I wish it was that obvious and easy to rank these days.

You will get some "purists" who will try to tell you that you should use a / on directories and no / on pages, but the reality is, as far as rankings go, it doesn't matter a bit which you decide to go with and for type-ins, consistency is better, imo.

Personally, everything I do now is totally extensionless, because no "normal people" have any clue there "should be" [according to some] a difference between a directory index URL and a page URL [keep in mind they're both pages and there's no rule I've seen that says the index of a directory [or the "default page" displayed when no specific page is requested] must be within the directory it's displayed for] and "clean" [meaning: extensionless, no trailing /] looks better to me and almost everyone [meaning: everyone outside of WebmasterWorld] I've asked about it, so in building "what visitors like better", I go with totally extensionless.

[lucy24]

You've bypassed the really important issue: Once you've decided to use something, redirect from all the forms you don't use. Don't just cross your fingers and hope nobody will ask for the wrong thing. If you're looking for ways to make search engines disapprove of you, Duplicate Content is far up the list.

If you use extensionless (aka "go back in the server and put some clothes on!") URLs, redirect any requests for URL plus trailing slash. That's explicit, external redirect, not silent rewrite. Do not accept requests with any extension. Your choice whether to redirect or just play dumb. ("example.com/page.html? I'm sure I have no idea what you're talking about.")

If you end each URL in a slash, do not accept requests for the same URL without a slash. Again, either redirect or send out an "ain't no such thing" 404 response.

Extensions are only meaningful if the extension matches the actual name of a physical file. Or a physical file that you used to have, if you've changed from html to php without otherwise changing the site structure. (Sure, let robots think you're operating strictly with static html. What they don't know won't hurt them.)

[martinibuster]

The previous members are correct in that there is no ranking advantage or disadvantage in using an extension like .html. However, what's been left (explicitly) unsaid is there is a practical reason why it may not be advisable to use an extension.

Using an extension like .html locks you into that file format. Should you move to another technology it may or may not be a pain to change the URL extension to something else and redirect the old and the new. It begins to become a larger problem once a few years down the road you upgrade to another technology and you find yourself redirecting to yet another URL file. And what if the IT person determines it's important to move to yet another CMS or shopping cart system? Scenarios like this have caused ranking issues in the past.

That's why if you have the opportunity to move from a legacy extension to a non-extension system to go for it. Doesn't matter if you choose / or non-/. As has already been stated, just redirect the chosen one to the other.

[lucy24]

Using an extension like .html locks you into that file format.

No more than going extensionless. It's the same rewrite either way.

RewriteRule ^paintings/mice/([a-jw]\w+)\.html /paintings/critters/critterlinks.php?subdir=mice&page=$1 [L,NS]

RewriteRule ^fun/CheesyNovel/chap(\d+)\.html /fun/CheesyNovel/cnlinks.php?chapter=$1&page=CheesyNovel [L]

"If I can do it, anyone can."

[robzilla]

So the options are www.mysite.com/category (without slash) or www.mysite.com/category/ (with slash) and of course the www.mysite.com/category.html

Assuming there are pages below the category level (and why else would you have categories), it would make most sense to me to use /category/, as a subpage might be /category/item or /category/subcategory/item. To me, this represents a 'natural' directory hierarchy, whereas /category.html (directly below the root, which seems illogical for a deeper category page) and /category (which can be confused for a page rather than a directory) aren't so much.

Finally, I would give preference to extensionless URLs for the reasons mentioned above. It doesn't hurt to have .html as long as you stick to it (preferably forever).

And I guess actually the home page as well would be included in the mix?

It wouldn't. Slash or no slash, the GET request would always be for "/", whether you include it or not. If you don't, your browser will do it for you (but may not show it).

[lucy24]

Assuming there are pages below the category level (and why else would you have categories), it would make most sense to me to use /category/, as a subpage might be /category/item or /category/subcategory/item

The hypothetical choice is among

/category/subcategory/item/
/category/subcategory/item
/category/subcategory/item.html

Like that.

[DansSitesRScrewed]

Glad I am getting some great information on this topic. As I mentioned before, there is a reason for my questioning, however it may need another thread to get to the bottom of it.

I am trying to determine a strange thing Google has been doing on a few of the sites on the server. Back when Panda was released, shortly after we noticed that for our category pages, the targeted keywords for a particular category were no longer pointing to the category page for that term. Google was associating really stupid pages for the terms, like the F&Q page or a product page with little or no relevance, etc. Bing however has indexed correctly and still does.

So this was happening on two sites and the common denominator was they are both Magento shopping carts. It is very frustrating because both of these sites were doing extremely well for the category pages, but now they are all screwed up. So since both sites are magento software, I have been trying to determine if maybe I am missing something on the coding end of things that google doesn't like.

If I wanted everyone to see examples, am I allowed to post urls? Or maybe start a new thread? I would really like some professional eyes that might see something that I am not.

Daniel

[mhansen]

I would ask what kind of site you have. eCommerce? Informational? And, what type of content is actually displayed on your category pages? You said:

because Google seems to be having issues with our category pages

If the above statement is the real issue you're having, have you considered it may be the content? We've found on sites that have weak category pages (WordPress, eCommerce) it could be that Google is filtering the category pages for duplicated content issues.

Just thinking out loud after seeing what you said about the issue on category pages.

MH

[martinibuster]

No URLs please.

No more than going extensionless...

I understand you and you're correct to a point. However here is what needs to be understood: The forward slash allows publishers to change technologies and keep the URLs the same. The actual URL could be changed to /index.html or /index.php but the URL structure remains the same forward slash: example.com/somefile/.

Using the forward slash gives you the flexibility of not having to use a redirect at all, ever. So Lucy, the code you published is rendered superfluous by using a forward slash because the underlying technology can change multiple times but a URL rewrite will not be needed, not at all, ever. No more rewrites.

This is and has always been an essential part of SEO, which is to keep the coding and technology as simple as possible. If you have a choice between doing something that may need a URL rewrite down the road at some point and doing it in a manner that does not, best practice is to choose the simpler option.

[JD_Toims]

The forward slash allows publishers to change technologies and keep the URLs the same. The actual URL could be changed to /index.html or /index.php but the URL structure remains the same forward slash: example.com/somefile/.

Using the forward slash gives you the flexibility of not having to use a redirect at all, ever.

Not sure I understand -- The only ways to do the preceding I can think of without an explicit rewrite are:

Create a directory with the same name as the file for every actual file on the server, place each file within it's "same name" directory and then rename every file on the site index.ext.

Create a directory with the same name as the file within it and set the directory index to the name of the file via directory specific .htaccess file.

---

Option 1 sounds like an absolute PITA to manage since every file on the entire site would have to be called index.ext.

Option 2 sounds even worse since you'd have to manage an .htaccess file in each separate directory.

---

In either case, you'd still have to redirect /path/file.ext to /path/ to avoid duplication between /path/ and /path/file.ext

---

I really can't figure out how what you're saying is possible any other way, so if there's something I'm missing that makes it so you can have all URLs end in a /, not use a rewrite and not create an individual directory for each individual file on the server, please let me in on the secret.

You can't get around a redirect being necessary to avoid duplication, unless you choose to make both versions of /path/ and /path/file.ext accessible and then rely on a canonical URL.

[martinibuster]

JD,
I anticipated your question and answered it already in the post:

The actual URL could be changed to /index.html or /index.php but the URL structure remains the same forward slash: example.com/somefile/.

Or put more explicitly,

1. If you change technologies and all of a sudden have to do this:

example.com/somefile/index.html

You can keep your original URL without a rewrite/redirect:

example.com/somefile/

2. If you change technologies and have to change URLs to this:

example.com/somefile/index.php

You can keep your original URL without a rewrite/redirect:

example.com/somefile/

3. If you move your site to WordPress or pretty much any CMS, you can keep your URLs without a rewrite or redirect.

4. Other advantages, which most SEOs haven't taken advantage of, is the ability to keep the URL simple by dropping the use of hyphens in the URL AND thereby making the URL easy to remember and communicate by word of mouth, pen on paper, or stylus on tablet device.

For example, I was interviewed by SearchEngineJournal.com as part of a PubCon Video Interview series. At the end of the interview I plugged my newsletter by speaking my domain name then saying, "forward slash newsletter."

Easy. Logical. Memorable. Elegant. Forward slash newsletter. Seriously, that's as cool as it gets. :)

5. A further benefit is of course links. Anything that makes it easier to remember or type the URL of any of your web pages is going to help others to link to your pages. URLs with multiple hyphens or random extensions are hard to remember. How is someone going to recall dot html or dot htm or dot php (not to mention hyphens) in order to speak, print or link to a complicated URL? Why would any web publisher put themselves at such a disadvantage?

Well, I know. The reason is because everyone else does it. Everyone else does it because sometime ten or eight years ago someone at Google said something about their algo understanding the meaning of keywords in URLs. Add that to the gospel of hyphens and the SEOs are out there doing something that to me resembles self-flagellation.

Good luck!
;)

[edited by: martinibuster at 3:17 am (utc) on Dec 30, 2013]

[aakk9999]

Anything that makes it easier to remember or type your URL is going to help others to link to your site.

I found that when people type (rather than copy) an URL when linking, they almost always forget the last slash. Which means you need to redirect no-slash to slash and lose some link juice this way.

I personally prefer pretty URLs with no slash at the end and I like to keep them as short as possible whilst still being meaningful to the visitor - I believe the CTR attributed to a well-crafted URL is an important factor.

[martinibuster]

aakk9999, you're absolutely right. That's a great reason to drop the slash.

In my case, I DO have the slash but that's my personal preference. I think that long term it's more future-proof, but that's splitting hairs and I won't quibble about that point.

To my eyes, a slash at the end looks more natural in the sense that it's expected, like a face with eyes, ears, mouth and nose. Without the slash it looks like something is missing and possibly it has that effect to others, particularly civilians. But again, we're splitting hairs, it's a personal preference and I don't wish to quibble about that point either.

Nevertheless, I still like having the ability of saying the URL without the slash and letting the rewrite do the rest. Ha! :P It is a trade-off I'm willing to live with. ;)

[lucy24]

1. If you change technologies and all of a sudden have to do this:
example.com/somefile/index.html

You can keep your original URL without a rewrite/redirect:
example.com/somefile/

2. If you change technologies and have to change URLs to this:
example.com/somefile/index.php

You can keep your original URL without a rewrite/redirect:
example.com/somefile/

example.com/blahblah/
=
example.com/blahblah/index.html
=
example.com/blahblah/index.php
=
example.com/blahblah/main.jsp
=
example.com/blahblah/any-random-garbage.xtn

DirectoryIndex any-random-garbage.xtn

BUT ONLY IF
example.com/blahblah/
is a real, physical directory.

Now, if it is a real, physical directory, then the final slash is a non-issue because Apache's directory-slash redirect takes care of it.

[JD_Toims]

1. If you change technologies and all of a sudden have to do this:

example.com/somefile/index.html

You can keep your original URL without a rewrite/redirect:

example.com/somefile/

2. If you change technologies and have to change URLs to this:

example.com/somefile/index.php

You can keep your original URL without a rewrite/redirect:

example.com/somefile/

Like I said, unless you rely on a canonical URL link on the /index.ext pages, you still have to redirect every page + extension to / -- If you only had one initially, you would have to update [add to] the redirect for a server-side technology change -- You *cannot* canonicalize /anything.ext to / without a redirect, unless you use <link rel=canonical> rather than a 301 -- No different than if you redirected everything to completely extensionless [meaning no / -- it's every bit as easy [actually easier] to strip everything as it is to leave a / and make sure if someone forgets a / it's redirected to, ime.] -- You cannot keep the original URL without some type of additional redirect, unless you rely on a band-aid [<link rel=canonical>] rather than a proper 301 to prevent duplication and possible link/ranking weight splitting.

5. A further benefit is of course links. Anything that makes it easier to remember or type the URL of any of your web pages is going to help others to link to your pages. URLs with multiple hyphens or random extensions are hard to remember. How is someone going to recall dot html or dot htm or dot php (not to mention hyphens) in order to speak, print or link to a complicated URL? Why would any web publisher put themselves at such a disadvantage?

Yup, why make people remember to end everything in a /, when the only people I've found who think "it looks wrong" or is "unnatural" are long-time SEOs/webmasters who have an "old school mentality" iow "old school way" of doing things? I can't see any reason, and if you watch TV [especially commercials] and the URLs presented a bit, you'll likely notice nothing presented to "the general public" ends in a / or .ext for some reason -- Go figure.

3. If you move your site to WordPress or pretty much any CMS, you can keep your URLs without a rewrite or redirect.

No you can't -- WP is completely "file system walk + rewrite driven" [slow as cold molasses] out-of-the-box when it uses "friendly" URLs [if "friendly URLs" are not present, it's /index.php?query_string_here], and even with "adjustments" it's completely driven by rewriting when "search friendly" URLs are used, otherwise all URLs would be: index.php?query_string_here -- WP URLs *absolutely* have to be rewritten to not be index.php, followed by a query_string. There's no other way to have "SE friendly URLs" with WP!

---

Sorry, MB, you're usually right on, but in this case I really have to disagree with what you're saying, because most of the things you're saying, including WP URLs ending in a / without a rewrite are impossible, unless people add a / to the end of the query_string for some reason, but that doesn't make sense or make it so people can change to WP [or other underlying technology] and have "exactly the same search friendly URLs" without a rewrite -- It's just not possible to do what you're saying, sorry.

[JD_Toims]

Back directly on-topic:

I am trying to determine a strange thing Google has been doing on a few of the sites on the server. Back when Panda was released, shortly after we noticed that for our category pages, the targeted keywords for a particular category were no longer pointing to the category page for that term. Google was associating really stupid pages for the terms, like the F&Q page or a product page with little or no relevance, etc.

It has nothing to do with your URLs -- Since Panda has been introduced, "obviously best" pages have been replaced in the rankings with "WTF? Why is this here?!" pages, for some reason, likely only "understood" by an algorithm/heuristic, imo -- I've seen numerous occasions where "the best" URL from a site for a query is replaced with "something else that doesn't really make sense", but it's not the URLs that are the issue.

[martinibuster]

...it's completely driven by rewriting when "search friendly" URLs are used...

Ugh. You're quibbling about backend rewriting. Has nothing to do with what I'm talking about. We're on the same page more than you realize, you're just coming at it from a different direction. What I'm referencing is a way to avoid having to REDIRECT legacy URLs to cover over legacy URLs to cover new URLs. That's all. That's it. It's just common sense. ;)

The point is, if you are using / or non-/ you can still move to WP and keep the same structure. Yes, there's internal rewriting going on BUT what the search engines see is the same URL, no REDIRECTS necessary from an old url to a new url. That's the point.

While Google/Bing have gotten good at dealing with these kinds of things, the point still stands that keeping it simple is the best strategy. That's the optimization part of SEO. ;)

[lucy24]

Exhibit A:

Yes, there's internal rewriting going on BUT

Exhibit B:

If you have a choice between doing something that may need a URL rewrite down the road at some point and doing it in a manner that does not, best practice is to choose the simpler option.

Exhibit C:

You're quibbling about backend rewriting. Has nothing to do with what I'm talking about.

:: lightbulb goes on ::

If you're not talking about rewriting, don't use the term "rewrite".

[buckworks]

found that when people type (rather than copy) an URL when linking, they almost always forget the last slash

Even copying is less likely to include the slash these days.

Safari 7.0.1 (Mac) leaves out the final slash when you copy an URL from the address bar.

Is that the way of the future?

[g1smd]

Once you decide to use URL rewriting, pages are best done with extensionless URLs and without a trailing slash. The matching RewriteRule RegEx pattern is then easy and unambiguous.

Real files such as images, css, js and so on, have extensions and it's therefore very easy to exclude those from the rewriting process.

Trailing slash denotes a folder or index page of a folder in the HTTP specs. A separate rule can set site-wide policy on how those are handled.

When designing a new site start with what the URLs will look like for each different type of page and for the various site assets, rather than simply exposing the internal database structure in the site URLs.