htaccess code issues

[jon22]

I need to remove the trailing slashes from the end of each url because google is treating them as individual pages, someone on the other forum told me the code has a lot of erros and to post it here.

<IfModule mod_dir.c>
  DirectoryIndex index.php
</IfModule>
<IfModule mod_gzip.c>
  mod_gzip_on    Yes
  mod_gzip_dechunk Yes
  mod_gzip_item_include file   \.(html?|txt|css|js|php|pl|jpg|png|gif)$
  mod_gzip_item_include handler  ^cgi-script$
  mod_gzip_item_include mime   ^text/.*
  mod_gzip_item_include mime   ^application/x-javascript.*
  mod_gzip_item_exclude mime   ^image/.*
  mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*
</IfModule>

RewriteEngine On
RewriteBase /
RewriteRule ^install/?$ install/index.php [QSA,L]
RewriteRule ^freshadmin/?$ freshadmin/$1 [QSA,L]
RewriteRule ^i/?$ library/phpThumb/index.php$1 [QSA,L]
RewriteRule ^img/(.+).jpg$ image.php?n=productimage&var1=$1 [QSA,L]
RewriteRule ^recommends/(.+)?$ linkfowarder.php?linkname=$1 [QSA,L]

#RewriteCond %{DOCUMENT_ROOT}/$1 -f
#RewriteCond %{REQUEST_URI} !^min/
#RewriteCond %{REQUEST_URI} !^install/
#RewriteRule ^(.+\.(css|js))$ min/index.php?f=$1 [L,NE]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)$ index.php?q=$1 [QSA,L]
RewriteCond %{HTTP_HOST} ^example\.org\.uk$ [OR]
RewriteCond %{HTTP_HOST} ^www\.example\.org\.uk$
RewriteRule ^car\-roof\-bars$ "http\:\/\/www\.example\.org\.uk\/roof\-bars" [R=301,L]

RewriteCond %{HTTP_HOST} ^example\.org\.uk$ [OR]
RewriteCond %{HTTP_HOST} ^www\.example\.org\.uk$
RewriteRule ^car\-roof\-boxes$ "http\:\/\/www\.example\.org\.uk\/roof\-boxes" [R=301,L]

RewriteCond %{HTTP_HOST} !^www\.
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]

[g1smd]

Rewrites with /?$ on the end of the RegEx pattern allow both URLs to directly serve content. Redirect "with slash" to "without slash" in a preceding (new) rule to prevent duplicate content.

Literal periods in patterns should be escaped.

Nearly every usage of .+ or .* in all of the rules should be replaced with something more specific.

The last three sets of condition and rule (those with R flag) should be listed BEFORE any of the rewrites.

The three redirects each have multiple syntax errors and are over complex.

Hyphens do not need to be escaped.

Nothing in the literal rule target needs to be escaped and the quotes are not needed.

In the single-page redirects, you do not need to test what domain was requested.

You do not need any of the IfModule container tags.

[jon22]

ok I cleaned code up.

Tried a few things to add the trailing slash because the majority of the urls are indexed like that. So all I need now is a redirect rule for that which is my next problem.

Also I noticed my home page has 2 trailing slashes now.

Even though there is a canonical tag with trailing slashes on each url Google still flag the non trailing slash even though they say a canonical tag is the best solution for SE's.

To try to force trailing slashes I tried

RewriteCond %{REQUEST_URI} !(\.[a-zA-Z0-9]{1,5}|/)$

in various places but keep getting 404 errors.

<IfModule mod_dir.c>
  DirectoryIndex index.php
</IfModule>
<IfModule mod_gzip.c>
  mod_gzip_on    Yes
  mod_gzip_dechunk Yes
  mod_gzip_item_include file   \.(html?|txt|css|js|php|pl|jpg|png|gif)$
  mod_gzip_item_include handler  ^cgi-script$
  mod_gzip_item_include mime   ^text/.*
  mod_gzip_item_include mime   ^application/x-javascript.*
  mod_gzip_item_exclude mime   ^image/.*
  mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*
</IfModule>

RewriteEngine On

RewriteBase /

RewriteCond %{HTTP_HOST} !^www\.
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)$ index.php?q=$1 [QSA,L]

RewriteRule ^car-roof-bars$ http://www.example.org.uk/roof-bars [R=301,L]
RewriteRule ^car-roof-boxes$ http://www.example.org.uk/roof-boxes [R=301,L]

RewriteRule ^install/?$ install/index.php [QSA,L]
RewriteRule ^freshadmin/?$ freshadmin/$1 [QSA,L]
RewriteRule ^i/?$ library/phpThumb/index.php$1 [QSA,L]
RewriteRule ^img/(.+).jpg$ image.php?n=productimage&var1=$1 [QSA,L]
RewriteRule ^recommends/(.+)?$ linkfowarder.php?linkname=$1 [QSA,L]

#RewriteCond %{DOCUMENT_ROOT}/$1 -f
#RewriteCond %{REQUEST_URI} !^min/
#RewriteCond %{REQUEST_URI} !^install/
#RewriteRule ^(.+\.(css|js))$ min/index.php?f=$1 [L,NE]

[g1smd]

The rule order is still wrong.

Considering only mod-rewrite code, the second block should be very last. After fixing that, the first two blocks should swap with each other.

URLs for pages should not end with a trailing slash. This is in the HTTP specs. Redirect to remove the slash.

The .+ and .* patterns should be replaced with something "more specific".

Dump the IfModule containers. They are not needed.

[jon22]

ok this time everything is working and the trailing slashes are stripped. Only I have left <Ifmodule> tags because the site returns a 500 internal server error without them.

Thank you so much for helping me.

<IfModule mod_dir.c>
 DirectoryIndex index.php
</IfModule>
<IfModule mod_gzip.c>
 mod_gzip_on  Yes
 mod_gzip_dechunk Yes
 mod_gzip_item_include file  \.(html?|txt|css|js|php|pl|jpg|png|gif)$
 mod_gzip_item_include handler ^cgi-script$
 mod_gzip_item_include mime  ^text/.*
 mod_gzip_item_include mime  ^application/x-javascript.*
 mod_gzip_item_exclude mime  ^image/.*
 mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*
</IfModule>

RewriteEngine On

#RewriteBase /

RewriteRule ^install/?$ install/index.php [QSA,L]
RewriteRule ^freshadmin/?$ freshadmin/$1 [QSA,L]
RewriteRule ^i/?$ library/phpThumb/index.php$1 [QSA,L]
RewriteRule ^img/(.+).jpg$ image.php?n=productimage&var1=$1 [QSA,L]
RewriteRule ^recommends/(.+)?$ linkfowarder.php?linkname=$1 [QSA,L]

#RewriteCond %{DOCUMENT_ROOT}/$1 -f
#RewriteCond %{REQUEST_URI} !^min/
#RewriteCond %{REQUEST_URI} !^install/
#RewriteRule ^(.+\.(css|js))$ min/index.php?f=$1 [L,NE]

RewriteRule ^car-roof-bars$ http://www.example.org.uk/roof-bars [R=301,L]
RewriteRule ^car-roof-boxes$ http://www.example.org.uk/roof-boxes [R=301,L]

RewriteCond %{HTTP_HOST} ^example.org.uk
RewriteRule (.*) http://www.example.org.uk/$1 [R=301,L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_URI} (.*)$
RewriteRule ^(.+)/$ http://www.example.org.uk/$1 [R=301,L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.+)$ index.php?q=$1 [QSA,L]

[jon22]

Oh yeah, what do you mean when you say "more specific"

[g1smd]

The rule order is still wrong. Now worse than before. Redirects must be listed before rewrites. Within each group the rules must be ordered from most specific to most general.

The .+ or .* pattern matches to the very end of the input. The parser then has to do a large number of back off and retry "trial match" attempts. This is very inefficient. You need a pattern that "matches from the left" and parses in one pass.

The 500 Error tells you that a module you want to use isn't installed. With the IfModule tags in place the whole block of code is being ignored. The result is that you have code there that never runs and you are never warned that is the case.

Literal periods in all patterns should be escaped.

[jon22]

So take all the [R=301] out and put them below #RewriteBase / ?

[g1smd]

Rules with protocol and hostname in target and/or [R] flag are redirects.

Rules with only a local path and file in target and without [R] flag are rewrites.

All the redirects must be listed first - and in order from most specific to most general.

After the redirects, list all of the rewrites - again in order from most specific to most general.

Literal periods in all patterns should be escaped.

Every rule should have at least the [L] flag.

[lucy24]

The 500 Error tells you that a module you want to use isn't installed.

If you take out the opening <IfModule... but forget the closing </IfModule> I think we can assume there will be some type of severe error. I do not propose to test this personally.

Quibble: If you're on shared hosting, I'm pretty sure the <IfModule etc doesn't necessarily mean the module isn't there at all. Otherwise we'd have to postulate an Apache installation without, say, mod_rewrite :o I think it only means that you personally do or don't have access to it, based on config file settings. In my case that seems to be how mod_security works.

Oh yeah, what do you mean when you say "more specific"

It means: give as much information as possible. For example, if you want your rule to apply only to named pages in the top-level directory, you'd say

RewriteRule ^[^/]+\.html et cetera (replacing "html" with whatever your real extension is)

so the Rewrite never even has to stop and look at the Conditions if the request is for an image or directory.

Things like this

RewriteRule ^car-roof-bars$ http://www.example.org.uk/roof-bars [R=301,L]
RewriteRule ^car-roof-boxes$ http://www.example.org.uk/roof-boxes [R=301,L]

can be collapsed into a single Rule:

RewriteRule ^car-roof-(bars|boxes)$ http://www.example.org.uk/roof-$1 [R=301,L]

or, if you want to be snarky about it,

^car-roof-b(ar|oxe)s$

;)