homepage Welcome to WebmasterWorld Guest from 54.237.98.229
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
htaccess security
How do you secure a website with htaccess?
pilot537780



 
Msg#: 4574858 posted 1:52 am on May 17, 2013 (gmt 0)

Hi All,

With some help from the forum, I got my domain redirects working. I am now trying to add some security.

After some research, I have come up with the following htaccess. I have added my comments/questions below the relevant sections. I am looking forward to your advice and suggestions.

Thank you,

Chris

# Enable basic rewriting
RewriteEngine On

order Deny,Allow


From what I have read, I understand you can have only one order line per htaccess. Is this correct? If it is, is "Deny,Allow" the way to go?

# secure htaccess file
<Files .htaccess>
deny from all
</Files>

# secure htpasswd file
<Files .htpasswd>
deny from all
</Files>

# ensure CHMOD settings for specified file types
# remember to never set CHMOD 777 unless you know what you are doing
# files requiring write access should use CHMOD 766 rather than 777
# keep specific file types private by setting their CHMOD to 400
chmod .htpasswd files 640
chmod .htaccess files 644
chmod php files 600

# Disable directory browsing
Options All -Indexes

<Limit GET POST>
allow from all
</Limit>


Am I correct in thinking this section can be deleted as it allows all anyway?

<Limit PUT DELETE>
deny from all
</Limit>

AuthName domain.com
AuthUserFile /dir/dir/dir/dir/file.pwd
AuthGroupFile /dir/dir/dir/dir/file.grp
Require valid-user


Is the AuthGroupFile line required? This was in the htaccess installed by my web host but when, I looked in the file.grp, there were no groups.

# Rewrite all domains not domain.com or not https to https://domain.com
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^domain\.com [NC,OR]
RewriteCond %{HTTPS} off
RewriteRule (.*) https://domain.com/$1 [R=301,L]

# Redirect index page to root
RewriteRule ^(.*)index\.(html|php)$ http://%{HTTP_HOST}/$1 [R=301,L]

# Block bad bots
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule ^.* - [F,L]

 

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4574858 posted 3:44 am on May 17, 2013 (gmt 0)

I understand you can have only one order line per htaccess.

Not at all. Who told you that? You can obviously only have one "Order" statment per ruleset. But within <Files> envelopes-- or within directory-specific htaccess files if you've got more than one-- changing order is one of the things you can absolutely do.

Now then...
RewriteEngine On
Put this statement immediately before your RewriteRules, and group the rules together. This is for your benefit; the server doesn't care.

order Deny,Allow
And again: put this statement immediately before the Deny and/or Allow rules it will apply to. I think Order directives are inherited-- meaning they'll continue to work in <Files> envelopes-- but don't take my word for it.

Options All -Indexes
Put all your Options statements near the beginning of your htaccess. And DO NOT repeat do not combine Options with plus/minus signs and options without. Apache docs [httpd.apache.org] are very emphatic about this.
Warning

Mixing Options with a + or - with those without is not valid syntax, and is likely to cause unexpected results

I suspect there is a special dispensation for All, but don't take chances.

If you say Options All you probably don't need Options FollowSymLinks-- required for mod_rewrite. But it won't hurt. Just be sure to use the right syntax.

# Rewrite all domains not domain.com or not https to https://domain.com

You mean "redirect", not "rewrite". Luckily the quoted rule does what you meant, not what you said ;)

# Redirect index page to root

These two rules are in the wrong order. The domain-name-canonicalization redirect should be the very last redirect. The /index.html redirect is generally the second-to-last.

# Block bad bots
RewriteRules that lead to [F] block 403 go before RewriteRules that create a redirect alone. Otherwise you're redirecting people who will later come back only to be locked out.

Now, personally I'd do this part in mod_setenvif, but that's me.

The [F] flag implies [L] so the [L] is not needed. It won't do any harm, so you can keep it if it makes you feel safer.

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4574858 posted 4:50 am on May 17, 2013 (gmt 0)

order Deny,Allow
From what I have read, I understand you can have only one order line per htaccess. Is this correct? If it is, is "Deny,Allow" the way to go?


per ruleset as lucy24 said...
when doing a "Deny from all" you want "Order deny,allow" and when doing a "Allow from all" you want "Order allow,deny".


# secure htaccess file
<Files .htaccess>
deny from all
</Files>
# secure htpasswd file
<Files .htpasswd>
deny from all
</Files>


i would just do this:
# secure .htaccess and .htpasswd file
<FilesMatch "^\.ht">
Order deny,allow
Deny from all
</FilesMatch>



chmod .htpasswd files 640
chmod .htaccess files 644
chmod php files 600


chmod is not an apache directive.
you have to do this from the command line or some other interface.
or were these lines intended as comments in the .htaccess file?


<Limit GET POST>
allow from all
</Limit>

Am I correct in thinking this section can be deleted as it allows all anyway?

it depends on the Order directive that applies (if any) and possibly the Deny directive that applies (if any).
as your .htaccess exists there is no Deny, so all is allowed by default.


Is the AuthGroupFile line required? This was in the htaccess installed by my web host but when, I looked in the file.grp, there were no groups.

if you are using a "Require group ..." directive then you need AuthGroupFile with one or more defined groups.
otherwise it is essentially an unused placeholder.


# Rewrite all domains not domain.com or not https to https://domain.com
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^domain\.com [NC,OR]
RewriteCond %{HTTPS} off
RewriteRule (.*) https://domain.com/$1 [R=301,L]

you'll have to describe your precise intention here.
for example you might want an end anchor in that second RewriteCond.


RewriteRule ^(.*)index\.(html|php)$ http://%{HTTP_HOST}/$1 [R=301,L]

using %{HTTP_HOST} will not canonicalize your hostname - it will simply reuse the requested hostname which may be non-canonical.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4574858 posted 6:01 am on May 17, 2013 (gmt 0)

it will simply reuse the requested hostname which may be non-canonical.

... or rather, that's what would happen if the rules were in the correct order ;)

http:/ /%{HTTP_HOST} is just the same as a bare / so you haven't gained anything; you've just used more space.

And wait a minute. Didn't you only just get through redirecting from http to https? How do you prevent an infinite loop if you're redirecting right back to http?

pilot537780



 
Msg#: 4574858 posted 10:29 am on May 20, 2013 (gmt 0)

OK, I have reworked my htaccess based on your advice. Is this better? See my comments/questions in the code.

# Disable directory browsing
Options All -Indexes

lucy24, I found this line of code on multiple websites about htaccess so there must be a special dispensation for All. That said, I would prefer to write it in the proper way but I don't quite understand what I need to do even after reading the Apache docs. Does each option need to be on its own line? Like:
Options All
Options -indexes
Options +FollowSymLinks
Or does it need to be like this (ie. All cannot be used with a + or - option):
Options -indexes +FollowSymLinks

Options +FollowSymLinks
lucy24, was this the line of code that is required for mod_rewrite?

# secure .htaccess and .htpasswd files
<FilesMatch "^\.ht">
Order deny,allow
deny from all
</FilesMatch>

phranque, thanks. That's much neater.

# ensure CHMOD settings for specified file types
# remember to never set CHMOD 777 unless you know what you are doing
# files requiring write access should use CHMOD 766 rather than 777
# keep specific file types private by setting their CHMOD to 400
chmod .htpasswd files 640
chmod .htaccess files 644
chmod php files 600

phranque, I found this code on a website (http://perishablepress.com/stupid-htaccess-tricks/) dealing only with htaccess. Are you sure this does not work in htaccess?

<Limit GET POST>
Order Allow,Deny
allow from all
</Limit>

<Limit PUT DELETE>
Order Deny,Allow
deny from all
</Limit>

AuthName domain.com
AuthUserFile /dir/dir/dir/dir/file.pwd
Require valid-user

# Enable basic rewriting
RewriteEngine On
RewriteBase /

Should I have RewriteBase in my htaccess?

# Block bad bots
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule ^.* - [F,L]

lucy24, I am looking into mod_setenvif. I have found a few interesting web pages about it. I also found you can use mod_security to block bad bots and I will have a look into that as well. I will probably start a new thread on this section later on.

# Redirect index page to root
RewriteRule ^(.*)index\.(html|php)$ https://domain.com/$1 [R=301,L]

lucy24, phranque, is this how I should do it then?

# Redirect all domains not domain.com or not https to https://domain.com
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^domain\.com [NC,OR]
RewriteCond %{HTTPS} off
RewriteRule (.*) https://domain.com/$1 [R=301,L]

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved