Directory Index, Folder Redirect and SEO

directoryindex redirects SEO

[cranfan]

I have a number of folders where I have nominated a file using Directory Index to deal with the scenario where a user just inputs www.mydomain.com/myfolder. This takes the user to www.mydomain.com/myfolder/ and displays the nominated page.

From an SEO point of view I am assuming it would be better if my canonical url was www.mydomain.com/myfolder/myfile.html. I am open to correction on this?
Reasons:
1) My file names are meaningful in terms of their content;
2) All the links within my site point to this url, not to “myfolder/”.

I have tried to set up a 301 Redirection to be applied after the Directory Index but without much success. I don’t know if it is a contradiction to try to do this or just my very limited knowledge of htaccess? I gathered the code on the web and modified it.

I also need to be sure that I don’t redirect requests for other files within “myfolder” !

I am testing with XAMPP on localhost.

This htaccess file is in “myfolder”:

DirectoryIndex myfile.html

Redirect 301 / http://localhost/mydomain/myfolder/myfile.html

#OR

RewriteEngine On
RewriteCond %{HTTP_HOST} ^/$
RewriteRule ^$ http://localhost/mydomain/myfolder/myfile.html [L,R=301]

This code is either looping or not displaying the page.

Thanks in advance.

[g1smd]

No. The canonical URL for a folder or the index page in a folder is www.example.com/folder/

Don't waste your time with fancy redirects that are not necessary.

The only one you do need is (www.)example.com/folder/index.html or .php to www.example.com/folder/ here.

[cranfan]

Thank you for your advice which I will adopt but with some regrets, I wish I had asked this sooner.

I am having difficulty getting the code redirecting the index pages to work. A lot of my page names are hyphenated so I have included that in the example in case it matters. Assuming I place this code in the same folder as the index file.

DirectoryIndex index-file.html

RewriteEngine On
RewriteCond %{THE_REQUEST} ^.*\/index-file\.html? 
RewriteRule ^(.*)index-file\.html?$ http://localhost/mydomain/myfolder/$1 [R=301,L]

I know some of this is moving into SEO rather than Apache but:

1) Does this mean that in order to optimise for SEO I should change all my internal hyperlinks likewise?

2) Equally, do I need to change any link rel="canonical" that currently point to myfolder/index-file.html?

3) Does an index file have any SEO priority over other files in the same folder? If not, I might change some of the index pages.

Lastly, how do you deal with the scenario where a url is requested with a trailing slash after a file name (either index or any other) e.g. mydomain/myfolder/file-name.html/. These are currently going to 404 errors. Can this be redirected with htaccess?

Appreciate your help. Thanks.

[g1smd]

Index file often has links to other files within folder.

For URLs ending in .html/ it is very easy to redirect to take that off.

# Strip / from .html/ or any extension at any folder depth.
RewriteRule ^(([^/]+/)*[^/.]+\.[^.]+)/$ http://www.example.com/$1 [R=301,L]

# You must NOT have a period within any real folder name.

Link to folder name followed by trailing slash for index pages in folders.

The redirect is also very simple.

# Redirect named index files to URL ending in /
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index-file\.html?
RewriteRule ^([^/]+/)*index-file\.html?$ http://www.example.com/$1 [R=301,L]

Add a ? after $1 if it also needs to strip parameters.

Add the usual non-www to www redirect AFTER the index redirect.

All this code goes in the root .htaccess file.

You need to change your rel=canonical to point to the shortest URL. Indeed, given
www.example.com/
www.example.com/index.php
www.example.com/index.php?page=root
the canonical URL should be the shortest one.

[cranfan]

Thank you, the feedback on this forum is excellent.

I have implemented both sets of code and everything woks well except I have a problem with the redirect if the index file is in a folder two levels up, then folder1 is being dropped.

e.g. mydomain/folder1/folder2/index-file.html

is redirecting as:

mydomain/folder2/

Is this easy to fix or can I use an htaccess file in folder2 to turn the rule off at this level? These are not important for SEO and I have not yet renamed my internal links.

[g1smd]

The code presented here, if added to the root .htaccess file, should not cause that effect.

There's either another rule elsewhere interfering with it or a typo somewhere.

Use the Live HTTP Headers extension for Firefox to investigate whether there's a multi-step redirect here. If there is there's multiple rules being invoked.

[cranfan]

Thank you for the response.

I have reduced my htaccess to just this in the root, no other htaccess:

Options +FollowSymLinks

RewriteEngine On

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*myindex-page\.html?

RewriteRule ^([^/]+/)*myindex-page\.html?$ http://localhost/websitefolder/$1 [R=301,L]

The code is exactly as I have it except for the substitution of folder and file names.

The effect I am getting is:

1 folder up:

http://localhost/websitefolder/folder1/folder2/

as expected

2 folders up:

http://localhost/websitefolder/folder2/

404 error

I have also reduced the page content to blank and turned javascript off.

Please could you give some htaccess code I can place in folder 2 to stop the Rewrite Rule at that level. At the moment the pages can’t be viewed.

[g1smd]

Use the Live HTTP Headers extension for Firefox to investigate whether there's a multi-step redirect here.

[cranfan]

If I enter:

http://localhost/websitefolder/folder1/folder2/myindex-file.htm

directly into the browser.

This is what I am getting in Live HTTP Headers.

Generator:

#redirect# GET /websitefolder/folder2/

Headers:

http://localhost/websitefolder/folder2/

GET /websitefolder/folder2/ HTTP/1.1
Host: localhost
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive

HTTP/1.1 404 Not Found
Date: Thu, 17 Nov 2011 13:26:24 GMT
Server: Apache/2.2.17 (Win32) mod_ssl/2.2.17 OpenSSL/0.9.8o PHP/5.3.4 mod_perl/2.0.4 Perl/v5.10.1
Vary: accept-language,accept-charset
Accept-Ranges: bytes
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=iso-8859-1
Content-Language: en

[cranfan]

I have been unable to resolve the above and I would like to switch the rule off at folder 2 level, at least until I find a solution.

The following is effective when placed in the local htaccess but no doubt there is a proper way to do this.

 RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*myindex-page\.html?
#
RewriteRule ^ [L]

Would appreciate the correct code that I should be using. Thanks.

[lucy24]

Oh, lord. Don't try to get a redirect to work locally. This level of experimenting has to be done on the real site. If necessary, make up a fake folder and run all your experiments there.

RewriteRule ^ [L] 

What is this intended to do? I kinda suspect you mean

RewriteRule myindex-page\.html? - [L]

meaning, basically, "don't do anything". Since the Condition says that the Rule will only apply to "myindex-page.html", put that part in the Rule itself, so your server doesn't have to stop and evaluate every single request it receives. That's in addition to, not instead of, the Condition as currently written.

[cranfan]

Thank you Lucy, that seems to work fine.

Maybe I won't need it on the live version.

[cranfan]

Given that all links to my home page should now end up at www.mydomain.com/ am I correct that it would be best SEO practice to set my internal links likewise?

So where I have:

href="../index.html"

I should amend this to:

href="../"

Or where I have:

href="index.html"

I should amend this to:

href="./"

Thanks.

[lucy24]

Neither.

/

That's all. The ../ construction is not SEO-harmful, but will lead to a big mess if you ever move things around. Save relative links for things that will always stay together as a package, like file+css+images groups.

Do as I say. Not as I do.

[cranfan]

Thanks Lucy.

Just so I am certain on this, you are saying that any link within my site to the home page should be href="/" regardless of the folder level it is at?

Using localhost this gives me [localhost...] which is an Apache admin page. No doubt you are going to tell me to test online which I don't have at the moment.

It worries me a bit having broken links for local testing?

[cranfan]

That should say:

http://localhost/

[lucy24]

Anything that begins in / is a site-absolute address, meaning that online it will be interpreted as www.example.com/ and so on. Conversely, anything ending in / is a directory and will normally give you the index file. (Somewhere along the line my browser decided to recognize this format even in local files, even when I'm not running MAMP. It won't take me to index.html but it will show me an index of the directory and that's good enough for testing.)

Can you install WAMP/MAMP? It is ridiculously easy-- I'm speaking as someone who will go to extreme lengths to avoid anything that requires a command-line interface-- and you can then set up a part of your home computer to behave exactly as if you were online. I think you can even make an htaccess, though I mainly use it for cross-directory link checking. It will only work for one "domain" at a time, but it's easy to change.

[g1smd]

You should never use the

../

construct in links to other pages, images or stylesheets that are pulled by the web browser.

I'm looking at a site where links on a page like

http://www.example.com/folder/page

are in the form

href="../otherfolder/otherpage"

.

This means: from "http://www.example.com/folder/page" strip off "page" and then strip off "folder/" and replace it with "otherfolder/" and then add "otherpage".

If you link to "/otherfolder/otherpage" with a leading slash, this means: add "/otherfolder/otherpage" on the end of the domain name. THAT is what you want.

[lucy24]

I say that you do want relative links within packages, because then if you move the package to a different location, you don't have to change anything. That's links beginning in an alphabetic character; if you're stepping back with ../ you may have an over-complicated package that you need to redesign anyway.

[g1smd]

What are you calling "relative"?

I say both domain-absolute ("

http://www.example.com/folder/page

" - with domain name) and root-relative ("

/folder/page

" - with leading slash) links are good, and page-relative ("

page

" or "

../

" or "

../page

" or "

../folder/page

" or "

../../../page

") links are bad.

[lucy24]

! You don't really want absolute links (http://www.example.com/etc.) within your own site do you?

Yes, I'm saying that links starting with alphabetic characters are sometimes the best and most appropriate way to do it. Unless you know beyond the shadow of a doubt that your site is carved in stone and nothing will ever be moved or rearranged. This assumption keeps a lot of htaccess files busy ;)

I may not know what directory a given file will be in next year, but I know that its images will always be right next door in images/ and its styles will be in thisgroup.css

[cranfan]

So if I understand this correctly before I publish my site live I could use Edit and Replace to replace any of "./" or "../" or "../../" with just / in each instance and in the live environment these links would still be valid?

The reason is not SEO but the load on the browser?

Lucy, I'm using XAMPP is WAMP/MAMP more involved?

[g1smd]

Not a stright replace unless the first folder after the / is actually off the root. Otherwise adjust the link so it begins

/first-level/second-level

counting from the root.

[lucy24]

I don't know XAMPP. All I know is that it would be hard to find a program less "involved" that MAMP ;) You just install it and run it like any old Mac program. No nasty command-line stuff. I kinda think WAMP behaves the same way. It's a little dialog box, and then your browser's address bar will say "localhost:some-port-or-other" instead of the long string of directories that you see if you open an html file locally without a fake server. The key difference is that once you've got this in place, the browser will go to the right place if a link starts in /

[cranfan]

Thanks guys.

Most of the folders are off the root, but you are correct, some I will have to amend.

I'll try MAMP on a spare Mac.