Newbie here - please grade my first .htaccess file

I have cut and pasted best on the web to create my first .htaccess file

[handsome tony]

Hello Everyone - I am a newbie and was wondering if anyone here with obviously greater experience and knowledge could review my first attempt at an .htaccess file? I would really appreciate it. Here is the code:

# disable directory browsing
Options All -Indexes

# BEGIN WordPress
RewriteEngine on
#
# Unless you have set a different RewriteBase preceding this
# point, you may delete or comment-out the following
# RewriteBase directive:
RewriteBase /
#
# if this request is for "/" or has already been rewritten to WP
RewriteCond $1 ^(index\.php)?$ [OR]
# or if request is for image, css, or js file
RewriteCond $1 \.(gif|jpg|css|js|ico)$ [NC,OR]
# or if URL resolves to existing file
RewriteCond %{REQUEST_FILENAME} -f [OR]
# or if URL resolves to existing directory
RewriteCond %{REQUEST_FILENAME} -d
# then skip the rewrite to WP
RewriteRule ^(.*)$ - [S=1]
# else rewrite the request to WP
RewriteRule . /index.php [L]
#
# END Wordpress

# Add Proper MIME-Type for Favicon / gzip
 AddType image/x-icon .ico
 AddType application/x-compress .Z
 AddType application/x-gzip .gz .tgz
 
# BEGIN Compress text files
<ifModule mod_gzip.c>
 mod_gzip_on Yes
 mod_gzip_dechunk Yes
 mod_gzip_item_include file \\.(html?|txt|css|js|php|pl)$
 mod_gzip_item_include handler ^cgi-script$
 mod_gzip_item_include mime ^text/.*
 mod_gzip_item_include mime ^application/x-javascript.*
 mod_gzip_item_include mime ^application/javascript.*
 mod_gzip_item_exclude mime ^image/.*
 mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*
</ifModule>
 
<ifmodule mod_deflate.c>
 AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css application/x-javascript application/javascript
 BrowserMatch ^Mozilla/4 gzip-only-text/html
 BrowserMatch ^Mozilla/4\\.0[678] no-gzip
 BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html
</ifmodule>
# END Compress text files
 
# BEGIN Expire headers
<IfModule mod_expires.c>
 ExpiresActive On
 ExpiresDefault "access plus 604800 seconds"
 ExpiresByType image/x-icon "access plus 2592000 seconds"
 ExpiresByType image/jpeg "access plus 2592000 seconds"
 ExpiresByType image/png "access plus 2592000 seconds"
 ExpiresByType image/gif "access plus 2592000 seconds"
 ExpiresByType application/x-shockwave-flash "access plus 2592000 seconds"
 ExpiresByType text/css "access plus 1209600 seconds"
 ExpiresByType text/javascript "access plus 1209600 seconds"
 ExpiresByType application/x-javascript "access plus 1209600 seconds"
 ExpiresByType application/javascript "access plus 1209600 seconds"
 ExpiresByType text/html "access plus 1 seconds"
 ExpiresByType application/xhtml+xml "access plus 604800 seconds"
</IfModule>
# END Expire headers
 
# BEGIN Cache-Control Headers
<IfModule mod_headers.c>
 <FilesMatch "\\.(ico|pdf|flv|jpg|jpeg|png|gif|swf)$">
  Header set Cache-Control "max-age=2592000, public"
 </FilesMatch>
 <FilesMatch "\\\\.(css)$">
  Header set Cache-Control "max-age=1209600, public"
 </FilesMatch>
 <FilesMatch "\\\\.(js)$">
  Header set Cache-Control "max-age=1209600, private"
 </FilesMatch>
 <FilesMatch "\\\\.(x?html?|php)$">
  Header set Cache-Control "max-age=604800, no-cache, must-revalidate"
 </FilesMatch>
</IfModule>
# END Cache-Control Headers
 
# BEGIN Turn ETags Off
<IfModule mod_headers.c>
  # properly handle requests coming from behind proxies
  Header append Vary User-Agent
  # remove ETags, it's important
  Header unset ETag
  FileETag None  
  Header unset Last-Modified
</IfModule> 
# END Turn ETags Off

[lucy24]

Options All -Indexes

On a driving test, this would be an instant fail. Apache [httpd.apache.org] says:

Mixing Options with a + or - with those without is not valid syntax, and is likely to cause unexpected results.

When Apache says "unexpected results", pay attention.

<FilesMatch "\\\\.(js)$">

What is the \\\\. intended to mean? Deduct one backslash for the literal period and you're left with three.

[handsome tony]

Dear Lucy

Thank you very much for your sobering reply.

With the

Options All -Indexes

I am attempting to hide my directory listing.

Could you tell me what it should be instead?

And with

<FilesMatch "\\\\.(js)$">

I am admittedly just copying something I found on another website. Should I change the four backslashes to just three for this line of code only, or for every instance?

I apologize for my lack of knowledge.

Thank you again for trying to help me.

[lucy24]

You don't need "All" since that is the default. If you say

Options -Indexes

then auto-indexing will be turned off, but no other options will change.

Should I change the four backslashes to just three for this line of code only, or for every instance?

It would be better to start by explaining what you're trying to do, and then code accordingly.

The parentheses in this line are also not necessary. Maybe your source started out with a pipe-separated list such as

\.(css|js)

If you're trying to match any and all javascript files, then all you need is

<FilesMatch "\.js">

The Files and FilesMatch directives match the end of the request; you don't have to say anything about what comes before.

Universal rule: Throw out any <ifmodule... envelopes. Not their content, just the envelope. You either have the module or you don't, and should code accordingly. You can replace the envelope with comments such as

# mod_setenvif stuff here

# mod_expires stuff here

if you find it useful. And speaking of comments: that list of RewriteConds, each preceded by its own comment, may be useful in a tutorial where you're going step by step. But I think most people would choose to collect all the comments and put them before the whole Rule-plus-Conditions package. Makes everything more readable.

Compare

# if this request is for "/" or has already been rewritten to WP
RewriteCond $1 ^(index\.php)?$ [OR]
# or if request is for image, css, or js file
RewriteCond $1 \.(gif|jpg|css|js|ico)$ [NC,OR]
# or if URL resolves to existing file
RewriteCond %{REQUEST_FILENAME} -f [OR]
# or if URL resolves to existing directory
RewriteCond %{REQUEST_FILENAME} -d
# then skip the rewrite to WP
RewriteRule ^(.*)$ - [S=1]
# else rewrite the request to WP
RewriteRule . /index.php [L]

vs.

# if this request is for "/" or has already been rewritten to WP
# or if request is for image, css, or js file
# or if URL resolves to existing file
# or if URL resolves to existing directory
# then skip the rewrite to WP

RewriteCond $1 ^(index\.php)?$ [OR]
RewriteCond $1 \.(gif|jpg|css|js|ico)$ [NC,OR]
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^(.*)$ - [S=1]

# else rewrite the request to WP
RewriteRule . /index.php [L]

No matter where you put the comments, leave a blank line after each RewriteRule so it's clear where one package ends and the next one begins. Otherwise it is easy to make a mistake and think that the Conditions apply to one Rule when really they apply to an earlier one.

Matter of fact, I am going to pause now. I made the mistake of looking a little closer at that pair of Rewrites-- which I realize you did not write yourself-- and it almost sent me into screaming hysterics.

Someone else will be along shortly (or possibly right now, because I type slow) and explain this in grownup terms ;)

[handsome tony]

Hi again, Lucy

Thank you for your feedback. I really appreciate it.

Here is what I am trying to do in layman's terms with my .htaccess:

1) Hide my directory listing so that people can't browse through it.

* It seems like we have this one solved, thanks for that.

2) This code is courtesy of jdmorgan, who posted it here:

[webmasterworld.com...]

and is intended to replace the default Wordpress .htaccess code.

* I assume this is still valid and a better way to go?

3) The next section is my attempt to use gzip compression in order to satisfy Google Page Speed. I am not sure if this code covers everything that needs to be covered, or even if it's correct. But I do show an improved Google Page Speed score since I started using it.

Furthermore, I have been trying to figure out a way to use .htaccess to compress files for Amazon Cloudfront CDN, but I have not been able to accomplish that. I had to turn off the CDN service because the files were not being gzipped when using Origin Pull. Perhaps this topic is better suited for a different thread.

4) The next section ExpireHeaders is supposed to optimize the caching functions by giving different objects different expiration dates. I am wondering if it's possible to optimize these settings so that it's no longer necessary to use a caching plugin for Wordpress?

5) Cache Control Headers - pretty much the same purpose as in the previous point 4, which is to say optimizing caching for speed and faster site loading times.

6) Not sure about Etags, other than everything I read said to turn them off.

In summary, I am trying to optimize my site so that it loads faster and makes Google happy. This is done through gzip compression and then caching on both the browser side and the server side. I would like to incorporate a Content Delivery Network (CDN) like Amazon Cloudfront, but am at a loss as to how to make gzip compression work with .htaccess rules.

Again, I am at the mercy of those with greater knowledge on this board and I very much appreciate any wisdom and advice you can share.

Thank you.

[lucy24]

This code is courtesy of jdmorgan, who posted it here

The problem with using code from jdmorgan is that he really, truly, deeply understands mod_rewrite. So if you cut-and-paste and then change one thing to fit your own installation, the whole thing is likely to fall apart. It's like, uhm, trying to achieve a flawless chicken tikka masala when you can just about, on a good day, manage to boil an egg without setting off the smoke alarm. (Someone will come along presently with a better analogy.)

Disclaimer: I'm not touching the gzip, deflate, expiration etc etc parts, for the rock-solid reason that I don't understand this stuff.

:: looking vaguely around for the people who do understand it, because I know they exist ::

I speak pretty fluent RegEx, and that's where it ends. But lemme see what the rewrites turn into if I give them a closer look.

[S=1]

Leave that kind of thing for jdmorgan. If the Rule is in the right location, a simple [L] will do. If it is in the wrong location, move it. Within each module, rules execute in order, straight down the page.

Now, line by line:

RewriteCond $1 ^(index\.php)?$

Wrong. If the request is for explicitly named "index.php", redirect to www.example.com/ alone. Only if the "index.php" is the result of a Rewrite do you keep it.

RewriteCond $1 \.(gif|jpg|css|js|ico)$

Wherever possible, rules that are constrained to a particular filetype should say so in the Rule itself. Otherwise, your server has to evaluate the Conditions for every single request it ever receives.

RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d

Boilerplate htaccess files always, always include this pair of lines. They are rarely appropriate. If the request is getting rewritten to a php page, let the php figure out if the user ought to be somewhere else. That's assuming there's a query string involved; the Conditions don't say.

Now, what if they ask for a file or directory that doesn't really exist? The Rule makes no distinction between pseudo-files-- ones that have no physical existence but are created by the program-- and ones that are simply garbage. The net result is that the user can request

www.example.com/anyoldcraphere.html

and the Rewrite will dutifully send them to index.php. This is OK if index.php then evaluates the request, establishes that there ain't no such page, and returns a 404. But if it simply carries on regardless, then you have Infinite URL Space. This is something that search engines will occasionally check for by requesting a garbage filename.

RewriteRule . /index.php [L]

Wrong. It should say .* because if the user requests the sitename by itself, then the request will have no content. In logs and mod_alias, a request for www.example.com/ will come through as / But mod_rewrite strips the slash after the domain name, so that leaves ... nothing.

The basic pattern for the universal rewrite goes

RewriteRule ^(([^/]+/)*([^/]+\.html)?)$ http://www.example.com/index.php?$1 [L]

meaning: If the user requests a page or directory, secretly change the request into a query string and rewrite to the index.php page where it will all be dealt with.

:: pause here to wait for other half of answer to come along later ::

[handsome tony]

Lucy, I am wondering if this looks any better?

# disable directory browsing
Options -Indexes

# BEGIN WordPress

RewriteEngine on
RewriteBase /
RewriteCond $1 ^(index\.php)?$ [OR]
RewriteCond $1 \.(gif|jpg|css|js|ico)$ [NC,OR]
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^(.*)$ - [S=1]
RewriteRule . /index.php [L]

# END Wordpress

# Add Proper MIME-Type for Favicon / gzip

AddType image/x-icon .ico
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz

# BEGIN Compress text files

mod_gzip_on Yes
mod_gzip_dechunk Yes
mod_gzip_item_include file \\.(html?|txt|css|js|php|pl)$
mod_gzip_item_include handler ^cgi-script$
mod_gzip_item_include mime ^text/.*
mod_gzip_item_include mime ^application/x-javascript.*
mod_gzip_item_include mime ^application/javascript.*
mod_gzip_item_exclude mime ^image/.*
mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*

AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css application/x-javascript application/javascript
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\\.0[678] no-gzip
BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html

# END Compress text files

# BEGIN Expire headers

ExpiresActive On
ExpiresDefault "access plus 604800 seconds"
ExpiresByType image/x-icon "access plus 2592000 seconds"
ExpiresByType image/jpeg "access plus 2592000 seconds"
ExpiresByType image/png "access plus 2592000 seconds"
ExpiresByType image/gif "access plus 2592000 seconds"
ExpiresByType application/x-shockwave-flash "access plus 2592000 seconds"
ExpiresByType text/css "access plus 1209600 seconds"
ExpiresByType text/javascript "access plus 1209600 seconds"
ExpiresByType application/x-javascript "access plus 1209600 seconds"
ExpiresByType application/javascript "access plus 1209600 seconds"
ExpiresByType text/html "access plus 1 seconds"
ExpiresByType application/xhtml+xml "access plus 604800 seconds"

# END Expire headers

# BEGIN Cache-Control Headers

<FilesMatch "\\.(ico|pdf|flv|jpg|jpeg|png|gif|swf)$">
Header set Cache-Control "max-age=2592000, public"
</FilesMatch>
<FilesMatch "\\\\.(css)$">
Header set Cache-Control "max-age=1209600, public"
</FilesMatch>
<FilesMatch "\.js">
Header set Cache-Control "max-age=1209600, private"
</FilesMatch>
<FilesMatch "\\\\.(x?html?|php)$">
Header set Cache-Control "max-age=604800, no-cache, must-revalidate"
</FilesMatch>

# END Cache-Control Headers

# BEGIN Turn ETags Off

Header append Vary User-Agent
Header unset ETag
FileETag None
Header unset Last-Modified

# END Turn ETags Off

[g1smd]

RewriteRule . /index.php [L]

Wrong. It should say .* because if the user requests the sitename by itself, then the request will have no content.

Actually, no, the " . " is quite OK here. Requests for

example.com/

will be handled by the

DirectoryIndex index.php

directive.

That's why the

RewriteCond $1 ^(index\.php)?$

code is also correct.

Of course, if you had used extensionless URLs the ruleset can be simplified to a single line of code.

[handsome tony]

Thanks to all for their replies. If someone could simply tell me what the critical issues are and what changes need to be made, that would be great. Once again, this is how my .htaccess looks at the moment:

# disable directory browsing
Options -Indexes

# BEGIN WordPress

RewriteEngine on
RewriteBase /
RewriteCond $1 ^(index\.php)?$ [OR]
RewriteCond $1 \.(gif|jpg|css|js|ico)$ [NC,OR]
RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^(.*)$ - [S=1]
RewriteRule . /index.php [L]

# END Wordpress

# Add Proper MIME-Type for Favicon / gzip

AddType image/x-icon .ico
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz

# BEGIN Compress text files

mod_gzip_on Yes
mod_gzip_dechunk Yes
mod_gzip_item_include file \\.(html?|txt|css|js|php|pl)$
mod_gzip_item_include handler ^cgi-script$
mod_gzip_item_include mime ^text/.*
mod_gzip_item_include mime ^application/x-javascript.*
mod_gzip_item_include mime ^application/javascript.*
mod_gzip_item_exclude mime ^image/.*
mod_gzip_item_exclude rspheader ^Content-Encoding:.*gzip.*

AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css application/x-javascript application/javascript
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\\.0[678] no-gzip
BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html

# END Compress text files

# BEGIN Expire headers

ExpiresActive On
ExpiresDefault "access plus 604800 seconds"
ExpiresByType image/x-icon "access plus 2592000 seconds"
ExpiresByType image/jpeg "access plus 2592000 seconds"
ExpiresByType image/png "access plus 2592000 seconds"
ExpiresByType image/gif "access plus 2592000 seconds"
ExpiresByType application/x-shockwave-flash "access plus 2592000 seconds"
ExpiresByType text/css "access plus 1209600 seconds"
ExpiresByType text/javascript "access plus 1209600 seconds"
ExpiresByType application/x-javascript "access plus 1209600 seconds"
ExpiresByType application/javascript "access plus 1209600 seconds"
ExpiresByType text/html "access plus 1 seconds"
ExpiresByType application/xhtml+xml "access plus 604800 seconds"

# END Expire headers

# BEGIN Cache-Control Headers

<FilesMatch "\\.(ico|pdf|flv|jpg|jpeg|png|gif|swf)$">
Header set Cache-Control "max-age=2592000, public"
</FilesMatch>
<FilesMatch "\\\\.(css)$">
Header set Cache-Control "max-age=1209600, public"
</FilesMatch>
<FilesMatch "\.js">
Header set Cache-Control "max-age=1209600, private"
</FilesMatch>
<FilesMatch "\\\\.(x?html?|php)$">
Header set Cache-Control "max-age=604800, no-cache, must-revalidate"
</FilesMatch>

# END Cache-Control Headers

# BEGIN Turn ETags Off

Header append Vary User-Agent
Header unset ETag
FileETag None
Header unset Last-Modified

# END Turn ETags Off

[g1smd]

No one can "simply" do that at all.

That's because the code you have there is not a program but instead is a site configuration file and there are an infinite amount of combinations of settings that could work - depending on what exactly it is you want to do.

This is why it is important that you understand what every line of code does so that you can configure it to work the way you want your site to work.

[Key_Master]

Hi tony, got your sticky mail.

I suggest you read up a little bit on regular expressions [sarand.com]. Once you learn about special characters and how they are used in the rules of your .htaccess file, this will make a lot more sense to you..

A literal period, (.) should be escaped (\.) because Apache use Perl compatible regular expressions (PCRE). In Perl, a period is a wildcard character that matches a single character.

In other words, "ab.cd" would match "ab[could be any character here]cd", whereas "ab\.cd" would literally match "ab.cd" because the period has been escaped.

Wildcard and other special characters (.*?{}()[]+ to name a few), should be escaped if they are intended to literally be used as characters.

Wildcard characters are escaped using a single backslash. The only time you would need a double backslash is if you wish to escape a literal backslash.

So

<FilesMatch "\\\\.(x?html?|php)$">

should be

<FilesMatch "\.(x?html?|php)$">

As far as the rest of the code goes, one conflict I see is:

ExpiresByType text/html "access plus 1 seconds"

<FilesMatch "\.(x?html?|php)$">
Header set Cache-Control "max-age=604800, no-cache, must-revalidate"
</FilesMatch>

This means html files may be cached and will expire in one second with a max-age of 604800 seconds but shouldn't be cached at all. Confusing, huh? Most browsers are probably going to ignore most of that and simply use whatever value it wants.

I would also reconsider unsetting the Last-Modified header. Googlebot and most browsers have very good support for this header.

Hope this helps. Back to vacation before I get caught "working". :)

[handsome tony]

Thank you very much for your replies. I am wondering if I should change all the multiple backslashes \\\\ to a single? If you look at this thread here, it seems like jdmorgan knows his stuff:

[webmasterworld.com...]

Also, what about changing the "1 seconds" to "0 seconds" here:

ExpiresByType text/html "access plus 1 seconds"

<FilesMatch "\.(x?html?|php)$">
Header set Cache-Control "max-age=604800, no-cache, must-revalidate"
</FilesMatch>

By doing so, can I leave the "no-cache" command in place?

[lucy24]

I am wondering if I should change all the multiple backslashes \\\\ to a single?

You should change them to the number of backslashes required by the context. That's the only possible generic answer.

If you look at this thread here, it seems like jdmorgan knows his stuff

Hm, that's kinda like saying the Taj Majal is a pretty nice tomb ;)

Seriously, I think you're doing way too much all at once. Most people's first htaccess looks something like this:

---

Options -Indexes

---

And then it grows with time as you find new needs and learn how to implement them. I think the second thing I put in my htaccess was a no-hotlinking routine, largely cribbed from a friend with a site about as small as mine.

[wilderness]

Seriously, I think you're doing way too much all at once. Most people's first htaccess looks something like this:

I agree with lucy.
My beginnings were with a few deny froms and anti-hot-linking.

[handsome tony]

Ladies and Gentlemen: I can't thank you enough for your replies. I apologize for my naivete - I am running a very busy website and trying to take its performance to the next level. In a nutshell, I want Google to look upon my site as being fast-loading and satisfying all of their Page Speed requirements. I thought it would therefore be easy for you to look at my code and say, "Your gzip compression is wrong." or "Your cacheing needs to be this instead of that if you want your pages to load at the fastest possible speed."

Are my expectations really that farfetched? Is it not possible to look once again at my code and point out some obvious shortcomings/tweaks?

Humbly Yours,

Tony

[wilderness]

tony,
FWIW, the forum charter [webmasterworld.com] provides two explanations which you apparently missed. (i. e., the 2nd and 3rd last).

In addition, sending sticky mails to every participant your able to locate is not a good practice either.

[handsome tony]

wilderness, am I supposed to get a Ph.D. in computer science overnight so that I can be treated as an equal on this board? Not everybody is a computer whiz like you.

The people I sent private messages to were all very gracious and helpful to me in their responses.

And I have tried to be as polite and deferential as I can on this thread.

Gee whiz!

[wilderness]

tony,
With all respect, you inquired for comments, and when they arrive, you seem to take them on-the-cheek, as opposed to advice and information.

FWIW, there's hardly any site on the internet that is absent UAG's or TOS'. Here at Webmaster World, it's a bit different because each individual forum has a different charter as well.

Many people simply click through UAG's and TOS' which are simply not read or complied with by newcomers(I've been guilty of it myself, as have many, many others).

[handsome tony]

OK, point taken - I will try to do better.

And I DO appreciate the replies and help from everyone!