.htaccess help - 403 error page issue

[war59312]

Well I have read all of:

A Close to perfect .htaccess ban list

Part 1, Part 2, And Part 3

So now I have a question.

For some reason it does not show my correct 403 error page.

Well I also tried adding with no luck:

RewriteRule !^/errors/403.php$ - [F,L]

But all I see in the browser is:

Forbidden
You don't have permission to access /sp3.php on this server.

Additionally, a 403 Forbidden error was encountered while trying to use an ErrorDocument to handle the request.

Instead of my correct error page.

Guess I am misunstanding how to use.

I also get the same result using wannabrowser.

Well here is my .htaccess file:

ErrorDocument 401 /errors/401.php
ErrorDocument 403 /errors/403.php
ErrorDocument 404 /errors/404.php

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://(www.)?example.com(/)?.*$ [NC]
RewriteRule .*\.(gif¦jp?g¦bmp¦png¦cab¦7z¦cmd¦txt¦log¦rar¦zip¦exe¦ex_¦dll¦dl_¦inf¦in_¦sys¦sy_)$ - [F,NC]

RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:[e-mail] [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
<snip>
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus [OR]
RewriteCond %{HTTP_USER_AGENT} ^ZyBorg
RewriteRule ^.* - [F,L]

Rewritecond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
RewriteRule \.php$ /errors/404.php

Options -Indexes
AddType application/cab .cab .sy_ .dl_ .ex_ .in_ .ch_
AddType application/cab cab sy_ dl_ ex_ in_ ch_
AddType application/x-httpd-php .foo

Any help with this would be great,
Will

[edited by: jdMorgan at 4:29 am (utc) on April 3, 2005]
[edit reason] Removed specifics oer TOS & trimmed. [/edit]

[sitz]

Not that this will change the current behavior, but (barring overriding reasons to the contrary) I'd suggest replacing this:

RewriteEngine on
RewriteCond %{HTTP_REFERER}!^http://(www.)?example.com(/)?.*$ [NC]
RewriteRule .*\.(gif¦jp?g¦bmp¦png¦cab¦7z¦cmd¦txt¦log¦rar¦zip¦exe¦ex_¦dll¦dl_¦inf¦in_¦sys¦sy_)$ - [F,NC]

With this:

RewriteEngine on
RewriteCond %{HTTP_REFERER}!^http://(www\.)?example\.com(/)? [NC]
RewriteRule \.(gif¦jp?g¦bmp¦png¦cab¦7z¦cmd¦txt¦log¦rar¦zip¦exe¦ex_¦dll¦dl_¦inf¦in_¦sys¦sy_)$ - [F,NC]

No sense in using those '.*' bits when you don't need to, and '.' characters need escaping to work as expected. And now, on to the meat of the issue...

You're getting that error because, well, Apache has been configured not to allow access to /errors/403.php. The easy things to check there are:

Does the webserver have read access to both DOCUMENT_ROOT/errors/ *and* to 403.php?

Can you request www.example.com/errors/403.php directly in your browser, or does that throw an error as well?

If you rename the .htaccess file (disabling it) and create a new one which only has the ErrorDocument 403 line in it and request a file to which you don't have access (one chmod'd 000, for instance), do you get the same error you get now?

Note that you may have another problem; this:

Rewritecond %{DOCUMENT_ROOT}%{REQUEST_URI}!-f
RewriteRule \.php$ /errors/404.php

...is an infinite loop waiting to happen, since '404.php' ends in '.php'. This *looks* as though it was added for debugging, but if you actually want this (not sure why you would; the ErrorDocument for '404' should take do so the same thing), I'd add:

RewriteCond %{REQUEST_URI}!^/errors/

above the current RewriteCond line. I removed this bit completely, and your .htaccess file (I downloaded it from the forum) works for me; I suspect that something else may be blocking the request. If you have read access to the ErrorLog, the reason for the 403 would be logged there.

One final note; in order for IE to display a server-supplied error message, it must be larger than 512 bytes (the actual size varies between error codes, but 512 bytes is large enough for all of them). For details on this behavior, see:

[support.microsoft.com...]
[support.microsoft.com...]

[war59312]

Hey,

All right I tried doing it your way. That is the first three lines you wanted me to switch.

Well now anyone can download my files. I just checked.

BTW the way it was before appears to work perfectly. You can only download files directly from my site. So no hotlinking and it appears to me to work perfectly.

So I could not for the life of me figure out how to do it your way. :(

And yes I can access [example.com...] fine and it disaplays correctly.

Um maybe I should have been clear.

I only see the wrong 403 error when you are a banned user.

I see the correct 403 error page when i try and access something i dont have access to like:

[example.com...]

Its only when you are banned. And by banned I mean like this:

order allow,deny
deny from ipaddress
allow from all

So all USER_AGENTs in my htaccess file are banned. But they dont see the correct error page.

Which I find very strange.

I dont see the log files. :(

Oh and about:

Rewritecond %{DOCUMENT_ROOT}%{REQUEST_URI}!-f
RewriteRule \.php$ /errors/404.php

Srry about that. Forgot to mention it. Yeah its a bug with my host.

If I try an access a php file that does not exist you get a No input file specified error. The above fixes that problem.

I did try it and I get a Internal Server Error 500 error.

So instead of getting that error it displays my 404 error page. Which is working like it should.

And yeah its over 1kb. :)

Well I think its an issue that banned users can not access my error pages. That simple.

So there must be away to exempt the error pages.

If you know what I mean.

Thanks for all your help,
Will

[sitz]

So all USER_AGENTs in my htaccess file are banned. But they dont see the correct error page.

If the .htaccess file you posted is in your document root, they would be banned; entries in a .htaccess file are inherited by subdirectories, so /errors/ inherits from the /.htaccess file, and since that's blocking access, the files in /errors/ won't be viewable either. Just add the following as the first RewriteCond line (right before the first HTTP_USER_AGENT RewriteCond):

RewriteCond %{REQUEST_URI}!^/errors/

[war59312]

I found the error:

[Sat Apr 2 22:16:14 2005] [error] [client ipaddress] client denied by server configuration: /home/user/example.com/errors/403.php

:)

Just saw your post.

Still get a Internal Server Error.

Log says:

[Sat Apr 2 22:20:29 2005] [alert] [client ipaddress] /home/user/example.com/.htaccess: RewriteCond: bad argument line '%{REQUEST_URI}!^/errors/'\n

[war59312]

OMG i feel like an idiot.

Simply:

order allow,deny
allow from all

In the errors folder.

lol God I'm such a dumb ass sometimes. lol

Well thanks a lot for your help,
Will

Well still have a question though.

Ok now I got it work.

Can I made it load a second 403 error page instead of the standard one?

Because I dont want ban users to see the normal 403 page. Instead I want them to see another page that says they have been banned or whatever.

But I sill want the normal 403 errors to be 403.php.

Hope u understand.

Thanks,
Will

[jdMorgan]

war59312,

Welcome to WebmasterWorld!

The procedure that sitz proposed was a debugging test only, which is why your anti-hotlinking stopped working. That was expected, since this was only a test to try to determine where the 403 problem was...

You can often save yourself a lot of trouble when using error documents combined with access control by adding a line similar to the following at or near the top of your .htaccess. Note that this has been "customized' for your current set-up:

RewriteRule ^(errors/[^/]+\.php¦robots\.txt)$ - [L]

This says, in effect, "If the request is for robots.txt or any custom error page, by-pass the remaining mod_rewrite rules in this .htaccess file." It prevents you from having to handle the robots.txt and error file cases in every access-control rule you add later. You may also wish to add your P3P policy file if you have one.

On another note, the sub-pattern "¦jp?g¦" in your rule should probably be "¦jpe?g¦". This would block either "jpeg" or "jpg" filetypes, since the "?" makes the "e" that precedes it optional.

Change all broken pipe "¦" characters above to solid pipe characters (usually Shift-\) before use. Posting on this board modifies them.

Jim

[war59312]

Ok thanks a lot.

Yeah I did not know about the P3P policy file but I am reading about it now. Thanks.

Just one question.

I want to ban txt files but I still need bots to be able to read:

[example.com...]

Atm it shows the 403 error page.

I thought:

RewriteRule ^(errors/[^/]+\.php¦robots\.txt)$ - [L]

Would fix that? L means its last right?

Thanks,
Will

[war59312]

Hey,

<snip>

Well, one problem is getright 5.2d can download everything. Since you can change the reffer to whatever you want. :(

Though luckly with defualt settings you get 403 forbidden. But still. :(

And you dont even have to click the link. You can just type in the download url and download without ever visting my site. :(

Well I could be mean. lol

RewriteCond %{HTTP_USER_AGENT} ^Mozilla [NC,OR]

Well it seems to be pretty useless to me. Since I can just change the refer to anything i want and its bypassed. :(

Is there a list of good user agents to allow? I only want to allow web browsers and thats it.

Thanks,
Will

[edited by: jdMorgan at 12:08 am (utc) on April 4, 2005]
[edit reason] Removed URL per TOS. [/edit]

[sitz]

Well one problem is getright 5.2d can download everything. Since you can change the reffer to whatever you want. :(

That would be the problem with instituting security based on unverified (in this case, inverifiable) data supplied by an untrusted user. Thus, referer-based security and user-agent security are inherently flawed. The only reliable methods of security are:

IP-based security, although even this isn't perfect, since proxies get involved;

authentication-based security (username/password) which, if you want to be REALLY paranoid, involve creating an HTTPS server so that usernames and passwords can't be sniffed on the network;

Cookie-based security (generally, but not necessarily used in conjunction with auth-based security). The trick with cookie-based security is to ensure that the contents of the cookie can't be tampered with. This generally involves encrypting the cookie prior to setting it, and decrypting the cookie supplied by the user. To be REALLY paranoid, one can also incorporate checksums into the cookiedata; something similar to what's described here: [modperl.com ]

[jdMorgan]

Yes, [L] should stop mod_rewrite processing for this pass through your .htaccess file. Note, however, that if you rewrite a URL, then httpd.conf and your .htaccess files will be re-invoked, so that access control and rewrites for the new URL can be checked.

Also note that [L] won't have any effect on mod_access directives, such as Allow from and Deny from. As a matter of fact, you cannot control the order that mod_access and mod_rewrite directives execute in. That depends on the order that the modules were loaded in Apache 1.x and on the module priority in Apache 2.x. In a "stock" server configuration, all mod_access directives will usually execute first, followed by all mod_rewrite directives.

Jim

[war59312]

oh ok

thanks,
will