Where to place htaccess file

htaccess location

[mikesz]

Hello and greetings,

I have a bit of confusion about .htaccess and where it does what on a forum website, both with the forum in the root and with it in a /forums folder (I have both models running)

The confusion is with the memberlist file and search engines. I don't trust that robots.txt will protect my memberlist adequately so I have added to my .htaccess file.

<Files memberlist.php>
order deny,allow
deny from all
</Files>

The question is if this block is in the .htaccess file in the webroot, does it protect a memberlist.php file if its in a subfolder and not in the root?

My understanding is that the .htaccess in the root is inherited to all subfolders but would like a confirmation that is true. I don't want to be adding stuff all over the place if I don't need it.

Now that I am looking at this, I realize that the example I am using doesn't really send the message I am trying to explain.

Here is a better example.

I block an IP address in the root .htaccess. If someone tries to access the /forums folder directly, i.e. mysite.com/forums/index.php. Does the block in the root folder prevent the IP address from accessing a subfolder?

Thanks in advance for your attention.

mikesz

[wilderness]

A child directory htaccess takes precedence over a parent/root folder.

IF, no child directory htaccess exists, than the parent/root folder htaccess applies to all lower child/sub-folders.

[jdMorgan]

It seems that simply typing the /memberslist.php and /forums/memberslist.php URLs into your browser should answer the question. If you get access, then the code did not work.

Jim

[mikesz]

Thanks for the replies. Thanks jd, that is why the memberlist is a bad example (I did test it and got a 404). I am more concerned about whether the IP address gets blocked in the subfolders which is the real problem I am not sure I am solving if I only put it in the root directory.

[wilderness]

I am more concerned about whether the IP address gets blocked in the subfolders which is the real problem I am not sure I am solving if I only put it in the root directory.

Do you have an htaccess file in the "subfolder" in question?
If so?

Than that subfolder htaccess overrides the root (parent) htaccess. Irregardless whether that subfolder htaccess contains or does not contain any denied IP range. The simple existence of a subfolder htaccess overrides the ENITRE ROOT HTACCESS, not just portions.

[mikesz]

Got it, thanks that tells me what I need to know.

[wilderness]

The simple existence of a subfolder htaccess overrides the ENITRE ROOT HTACCESS, not just portions.

Thought I should clarify this further, in the event that another comes along and takes it out of context.

The simple existence of a subfolder htaccess overrides the ENITRE ROOT HTACCESS, as related to the contents of the subfolder, not just portions.

[Caterham]

A simple existence of a .htaccess file, which is a per-directory configuration file, would do nothing without directives defined.

Each module can define how the per-server (main server, <virtualhost> section) and per-directory (<Directory>, .htaccess, <DirectoryMatch>, <Files>, <FilesMatch>, <Location>, <LocationMatch> sections) configuration is being merged. What is merged? A configuration structure which is created when the configuration is read and a directive provided by a module was found and the corresponding command (provided by the module which provides the directive) runs (that is not the runtime "run", it's just translating the directives and values into something to work with later). The configuration structure is merged either by a merging function provided the module or - fallback - by core. Depending upon a merging function, some directives override prior values of a directive, others accumulate.

If no new config structure for module x was created (i.e. no directive for module x was found in the walked section or parsed .htaccess file) nothing can happen with the merged configuration for module x at all because there is no config structure to merge with (see

ap_merge_per_dir_configs()

in server/config.c).

[wilderness]

Ok I exaggertated for the benefit of comprehenision.

"RewriteEngine off"

No directives?

[mikesz]

Ok, so if a deny from some IP is in the root .htaccess and no other .htaccess exists in any child/subfolders, does the deny from also apply to the contents of the subfolder? Conversely, if I have a .htaccess in the subfolder does it too require the deny from statement to also be present there to prevent access by the IP address?

[wilderness]

Ok, so if a deny from some IP is in the root .htaccess and no other .htaccess exists in any child/subfolders, does the deny from also apply to the contents of the subfolder?

yes, and no (there are alternatives to Deny from).

Conversely, if I have a .htaccess in the subfolder does it too require the deny from statement to also be present there to prevent access by the IP address?

yes, and no (there are alternatives to Deny from).

As an aside, this really shouldn't be an issue, as it's a similar method to what is used in the application off CSS within html and/or external sheets.

[jdMorgan]

It would perhaps be more accurate to say that directives in a lower-level .htaccess file can override settings in a higher-level .htaccess file, and that the override will apply only to files in lower-level .htaccess file's subdirectory and its subdirectories.

So if there is a line in /.htaccess
Deny from 123.45.67.89

and another line in /subdir/.htaccess
Allow from 123.45.67.89

Then assuming for simplicity that no other applicable directives are present, a visitor from the IP address 123.45.67.89 cannot access example.com/index.html, but can access example.com/subdir/index.html or example.com/subdir/sub-subdir/index.html

See [httpd.apache.org...] and the links on that page for more info.

Jim

[Caterham]

"RewriteEngine off"

No directives?

That's a directive from mod_rewrite but not 'no directives'. Hence there is a new new config structure for this module and mod_rewrite's per-dir merging function runs.

[mikesz]

In doing some follow up testing on this issue, I seem to not be able to block an IP address using the described example with my own IP address. The memberlist block does work. Is it possible for the ISP to disable deny/allow only?

[Caterham]

Is it possible for the ISP to disable deny/allow only?

If AllowOverride limit is not set the directives order, allow and deny have no effect if placed in a .htaccess file.

[mikesz]

Oh, that's good to know, thanks. So far, I have discovered that cPanel IP block seems to be implicated in it. From what I see, if hijacked the Deny then does nothing with it so its like you don't have a block at all?

[jdMorgan]

Do you have an "Order" statement in your own code that pertains to your IP address Allow and Deny directives? The Order directive for your memberslist only applies to the memberlist files, because it is encloosed in at <Files> container.

This might go faster if you posted all Order, Allow, and Deny directives in your .htaccess file(s) as well as any <containers> in which they are enclosed. Also, tell us the location of your .htacess file(s), so that their scope and precedence are clear.

Jim

[mikesz]

The .htaccess is in the root folder. Here is the contents of the file minus a few hundred IP addresses:

Options +FollowSymLinks -MultiViews
RewriteEngine On
RewriteBase /

# custom error documents
ErrorDocument 401 /401.shtml
ErrorDocument 403 /403.shtml
ErrorDocument 404 /404.shtml
ErrorDocument 500 /500.shtml

<IfModule mod_rewrite.c>
RewriteCond %{HTTP_HOST} ^mysite.com\.com$ [NC]
RewriteRule ^(.*)$ [mysite.com...] [R=301,L]
</IfModule>

RewriteCond %{HTTP_USER_AGENT} ADSARobot地h-ha地lmaden地ktuelles再narchie地mzn_assoc再SPSeek再SSORT
<snip>
名idow名ISEbot名WWOFFLE寺-Tractor回Xaldon\ WebSpider名UMPUS合enu合GET后eus.*Webster后eus [NC]
RewriteRule ^.* - [F,L]

<Files *>
order deny,allow
deny from 64.202.189.170
deny from 68.178.254.168
deny from 4.79.181.
deny from 12.208.109.
deny from 12.20.154.12
deny from 12.180.246.7
deny from 24.206.236.63
deny from 24.21.179.128
deny from 24.217.81.20
deny from 24.31.28.222
deny from 24.4.31.78
allow from all
</Files>

<Files memberlist.php>
order deny,allow
deny from all
allow from vbplusme.com
allow from localhost
</Files>
<Files ~ "^\.">
Order allow,deny
Deny from all
</Files>

<Files ~ "^\.\.?$">
Order allow,deny
Allow from all
</Files>

[edited by: jdMorgan at 1:14 pm (utc) on April 17, 2009]
[edit reason] Snipped long UA list, fixed side-scroll. [/edit]

[jdMorgan]

I can never remember exactly what the "rules" are, but I have never had any luck getting multiple Order directives to work in .htaccess files. I suggest that you use a single "Deny,Allow" up at the top and outside of any <containers>, and then adjust the "logic" of your access control code blocks to work based on that Order.

With Order Deny,Allow, the Denies are evaluated before the Allows, Allows can therefore override Denies, and any request not matching any Deny is permitted.

For example, this allows you to Deny a particular robots' IP address block so that it cannot fetch *any* files, but to use an Allow directive to allow *all* robots to fetch robots.txt. In this way, all robots can discover whether they are allowed to spider your site, but the one whose IP address you blocked cannot fetch any files other than robots.txt.

As a result, you should also include an Allow so that *all* user-agents (including those you've blocked by user-agent or IP address) can fetch your custom 403 error page. Failure to do so constitutes a self-inflicted denial of service attack, since any 403 response to a completely-denied client will result in an attempt to serve the custom 403 error page, which will result in a second 403, leading to another attempt to serve the custom 403 error page, leading to another 403 -- ad nauseum.

The easiest way to do this is to use mod_setenvif:
[code]
SetEnvIf Request_URI ^robots\.txt$ allowit
SetEnvIf Request_URI ^403\.shtml$ allowit
.
.
.
Allow from env=allowit
[code]
Jim

[mikesz]

The .htaccess is in the root folder. Here is the contents of the file minus a few hundred IP addresses:

Options +FollowSymLinks -MultiViews
RewriteEngine On
RewriteBase /

# custom error documents
ErrorDocument 401 /401.shtml
ErrorDocument 403 /403.shtml
ErrorDocument 404 /404.shtml
ErrorDocument 500 /500.shtml

<IfModule mod_rewrite.c>
RewriteCond %{HTTP_HOST} ^mysite.com\.com$ [NC]
RewriteRule ^(.*)$ [mysite.com...] [R=301,L]
</IfModule>

RewriteCond %{HTTP_USER_AGENT} ADSARobot地h-ha地lmaden地ktuelles再narchie地mzn_assoc再SPSeek再SSORT
名UMPUS合enu合GET后eus.*Webster后eus [NC]
RewriteRule ^.* - [F,L]

<Files *>
order deny,allow
deny from 64.202.189.170
deny from 68.178.254.168
deny from 4.79.181.
deny from 12.208.109.
deny from 12.20.154.12
deny from 12.180.246.7
deny from 24.206.236.63
deny from 24.21.179.128
deny from 24.217.81.20
deny from 24.31.28.222
deny from 24.4.31.78
allow from all
</Files>

<Files memberlist.php>
order deny,allow
deny from all
allow from mysite.com
allow from localhost
</Files>
<Files ~ "^\.">
Order allow,deny
Deny from all
</Files>

<Files ~ "^\.\.?$">
Order allow,deny
Allow from all
</Files>

[mikesz]

I must have not done what I thought and have another copy of my htaccess post? I thought I was fixing the one that broke out to the side. Anyway, thanks for the ideas, I will rewrite it. Just changing the order inside the containers is breaking with a Server 500 for some reason. I will try your suggestion in any event.

thanks very much.

[mikesz]

Hello Jim,

I modified the .htaccess and removed the container and put the order at the beginning of the IP list, got a Server 500 so I removed the order statements for the IP addresses and everything worked correctly. Seems like cPanel is taking care of the order directive in this system. It never worked like that in other cPanel systems I have used so its a total surprise to me.

[jdMorgan]

That is odd, since the only prerequisite to using "Order" is to have "AllowOveride Limit" set in the server configuration. But "Allow" and "Deny" also require "AllowOveride Limit" to be set, so if it wasn't, then they should have failed too.

You might want to dig into your server error log to see what the specific cause of the 500-Server Error was.

But I'm glad you've got it working, anyway. :)

Jim

[mikesz]

AH, the Server 500 Error, it was caused by exactly what you mentioned in your suggestion, "self-inflicted denial of service attack". Left to its own devices, cPanel inserts the following:

<Files 403.shtml>
order allow,deny
allow from all
</Files>

Without it, as soon as you add an IP to the list using cPanel, you get the Server Error 500. I didn't know that is what cPanel did until I cleared the whole .htaccess file and ran a cPanel test. It appended the IP address to the bottom of the file but added this block before it. When I added it to the broken .htaccess file, the Server Error 500 went away.

I really appreciate your help with this, without your suggestion, I'd still be thrashing around trying to find the problem. Thanks very much.