Welcome to WebmasterWorld Guest from 50.19.0.90

Message Too Old, No Replies

URLs end with slash and without - how to fix ?

     
11:12 am on Sep 17, 2013 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 24, 2003
posts: 115
votes: 0


URLs end with slash and without - how to fix ?

From GWT,
the report showing duplicate header reveals google indexed both version.

Should i do something to avoid being consider as duplicated content ?

Thanks.
11:51 am on Sept 17, 2013 (gmt 0)

Full Member

5+ Year Member

joined:Dec 27, 2006
posts:311
votes: 0


Use rel=canonical metatag.

If you have 2 pages with identical content
mywidgets.com/product-name-widget-1
mywidgets.com/product-name-widget-1/

and the correct link/page is
mywidgets.com/product-name-widget-1

on both pages put (do not forget to add http:..)
<link rel="canonical" href="mywidgets.com/product-name-widget-1" />

In this way you tell to search engine that correct version is that without "/" at the end.
12:06 pm on Sept 17, 2013 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:10542
votes: 8


google still calls the link rel canonical a "suggestion".

http://support.google.com/webmasters/answer/139394 [support.google.com]:
Adding this link and attribute lets site owners identify sets of identical content and suggest to Google: "Of all these pages with identical content, this page is the most useful. Please prioritize it in search results."

pretty please?

the most robust solution is to redirect the non-canonical version of the url to the canonical version of the url using a 301 status code.
you simply don't serve a 200 OK and content to requests for non-canonical urls.
this solution doesn't require any guesswork or unnecessary processing by google.
7:48 pm on Sept 17, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12692
votes: 243


If you want readers to think of it as a page, use the without-slash form. If you want readers to think of it as a directory (frankly this doesn't seem likely), use the with-slash form.

And now go yell at the designers of your CMS for not enforcing a single URL format.
9:32 pm on Sept 17, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Nov 11, 2007
posts:769
votes: 1


It's just personal preference. Either works, but the key is to allow only one OR the other, never both. And I agree w/ Frank... using 301 redirects to correct the canonicalization issue is a much better solution than using rel="canonical".

If you fix the issue w/ 301 redirects (which have been the preferred way to correct canonical issues a LOT longer than rel="canonical") then users will never again see a non-canonical URL in their browser even when they click on a non-canonical link. They are immediately redirected to the canonical form of the URL and the browser's address bar is updated to reflect the canonical URL. A big advantage of this is that virtually all links created in the future will point to the canonical URL since 99.99% of the links created on the web are created by navigating to the page to be linked, copying the URL from the browser address bar, and pasting it into HTML, code, or a CMS to create a link.

If you fix the issue w/ rel="canonical" then users will likely continue to see non-canonical URLs in their browser. This means they will likely continue to build links to the non-canonical URL. IMO rel="canonical" should be used only as a last resort if you cannot create 301 redirects or if creating redirects to solve the issue is simply too complex (such as is the case on lots of ecomm sites using many query string parameters in the URL and simply changing the order of the parameters - not even the values - creates a new URL.)

Personally, if your URLS are all extensionless, I like having ALL URLs end in "/" instead of ending with what "appears" to be an extensionless filename.

I think:

example.com
example.com/level-1
example.com/level-1/level-2
example.com/level-1/level-2/level-3

just looks dumb. All internal nodes in the URLs sometimes looks like a file (e.g. "level-1" appears to be a file in /level-1) and sometimes looks like a folder ("level-1" appears to be a folder in /level-1/level-2).
10:36 pm on Sept 17, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


Use
example.com/folder/
for folders
and
example.com/page
for pages.

A URL such as
example.com/stuff/
with a trailing slash gives a clue that there may be more levels.

Make sure the internal navigation always links to the correct form.
Redirect requests for the incorrect form.
12:21 am on Sept 18, 2013 (gmt 0)

Full Member

joined:Apr 26, 2012
posts:327
votes: 8


I think standard wise, files are supposed to be without a slash and folder with one. Practically, it's a mixed bag to whole follows this and who doesn't, and as far as I can tell Google can tell the difference between a page and a folder (or a category, as it usually is), even if both use a slash.
1:55 am on Sept 18, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12692
votes: 243


I should hope that g### knows the difference between an URL and a filepath :) But it can't tell whether two different URLs yielding identical content represent one physical file or two.

We say that final / means a directory. But the word "directory" is really shorthand for "the index file of the specified directory". Even an auto-index isn't really the raw, machine-level index of a directory; it's an html file created to look that way.
7:14 am on Sept 18, 2013 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 24, 2003
posts: 115
votes: 0


>>redirect the non-canonical version of the url to the canonical version of the url using a 301 status code.

can you teach me how to do it ?
The adjustment apply to sitewide, or i have to set it page by page ?
i am a little dumb.
7:38 am on Sept 18, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


You should add a rule to redirect requests for example.com/folder/index.php (and .htm, .html) to example.com/folder/

When /folder/ exists, a request for example.com/folder should already be automatically redirected to example.com/folder/

You need to add a rule so that when example.com/page/ or example.com/folder/page/ is requested, the request is redirected to example.com/page or example.com/folder/page respectively.
7:45 am on Sept 18, 2013 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:July 19, 2013
posts:1097
votes: 0


can you teach me how to do it ?

There is a recent thread on this exact subject in the Apache Forum, which is a much better place for the "how to" part to be explained. You can find the thread here: [webmasterworld.com...] Feel free to hop into the current thread or start another one and I'm sure someone will try to give you a hand working through things even though we don't usually code for people for a number of reasons.
1:44 pm on Sept 18, 2013 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:10542
votes: 8


the trailing slash redirect is partly handled by mod_dir:
http://httpd.apache.org/docs/current/mod/mod_dir.html [httpd.apache.org]
9:14 pm on Sept 18, 2013 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12692
votes: 243


The trailing-slash redirect turned out to be part of the problem in the thread jd linked to. Over there, the OP's remove-slash redirect included one URL that was a real, physical directory, so mod_dir kicked in and created an infinite loop. But everywhere else-- and, I think, in the present thread too-- the URL are created out of whole cloth by the CMS.

Ideally you want a pattern of redirects that doesn't involve a !-d check at every stage.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members