enabling a 404 error code in .htaccess

Going from a 403 to a 404

[Michel Samuel]

My current error codes in my .htaccess file look like this...

Order Deny,Allow
deny from 255.255.255.255
<Files 403.shtml>
Allow from all
</Files>
RewriteEngine on
RewriteCond %{HTTP_REFERER} asite\.org [NC]
RewriteRule!403\.shtml$ - [F]

For both sections I just want to give a 404 error and not display a page at all. (bandwidth has become a big issue for me.)

I'm not finding any informations on the order deny section.
But I have done the rewrite rule below and it isn't working.

RewriteRule!ErrorDocument 404 [mysite.com...]

I have tried but I'm not understanding how this works.
--------

[g1smd]

>> RewriteRule!ErrorDocument 404 http://mysite.com/404 <<

A RewriteRule turns a request for one URL into a request for another URL or for a filepath, either as an internal rewrite or as an external (301 or 302) redirect.

An ErrorDocument directive specifies what file to serve when each particular error number occurs in the server. You must NOT include the http:// or point to another server, otherwise you will NOT get the correct HTTP error code sent back (404 will be returned as 302 in fact).

You cannot mix up RewriteRule and ErrorDocument directives. They are completely different things.

[Michel Samuel]

Can I just remove the rewrite then and do this.

ErrorDocument 404

[jdMorgan]

You must serve a file in response to an error. However, that file can be small, or even blank.

The correct format for ErrorDocument is

ErrorDocument 403 /403.html
ErrorDocument 404 /404.php

Jim

[Michel Samuel]

Ok that means my file must look like this.

Order Deny,Allow
deny from 255.255.255.255
<Files 403.shtml>
Allow from all
</Files>
RewriteEngine on
RewriteCond %{HTTP_REFERER} asite\.org [NC]
errorDocument 404 /erreur404.hml

BUT...

2 questions.

1) How do I produce the 404 error for my IP Deny section?

2) Does simply saying errorDocument 404 cause the browser to register the actual 404 sever error? (I ask because I'm trying to stop several auto-surfers and bots. A human may not actually see the document)

[g1smd]

>> ErrorDocument 404 /erreur404.html <<

This says that when a file is not found on the server, to instead serve the file at /erreur404.html along with a 404 HTTP status code in the HTTP header.

[g1smd]

>> RewriteCond %{HTTP_REFERER} asite\.org [NC] <<

If you have a RewriteCond than you'll need a matching RewriteRule after it.

Be aware that your unanchored asite\.org will allow "asite.org" to appear anywhere in the referring URL, not just in the domain name.

That is, a URL like www.nasty-domain.com/some-folder/whatever-asite.org-blah-blah/ will get around your "security".

[g1smd]

You don't serve a 404 error to denied users, they get a 403 instead.

The 404 error is for pages that do not exist, not for pages that are simply denied.

[Michel Samuel]

This stuff always gets complicated quickly.
OK.. I am breathing deeply.

----------------------

I am lost again.
How do I produce the 404 error?

Currently in my rewrite rule I am flag it as " -F " (403 forbidden)

----------------------

One of the things I have done in my deny section is ban the addresses of my big problem domains. I want to return the same 404 error to them to. Is this possible?

---------------------
can I just do this for my 404 page and nothing more.

<title>HTTP 404 Supprimé</title>

And if I am displying a custom 403 page can I just put the same http status code 404 in that page?

[edited by: Michel_Samuel at 11:00 pm (utc) on May 30, 2007]

[Michel Samuel]

I am researching the http status codes and it is becoming obvious I have to be a programer to make the page.

This is just too much difficulty for me. It is like trying to learn english as a second language. And the teacher explains to you what an english pronom is but he does it in french and never bothers to tell you the words.

Thank you much for the help once again. I might appear to be upset but I genuinely appreciate the efforts.

[jdMorgan]

Michel,

First of all, you cannot produce a 404 for your deny section. The response for the deny section is dictated by the server.

What you can do is to reduce the size of your 403 error page, something like:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta http-equiv="Content-Language" content="fr-FR">
<meta name="robots" content="none">
<title>Access Denied</title>
</head>
<body text="#000000" bgcolor="#FFFFFF" link="#000099"><br>
<center>
<h1><font face="Arial,Helvetica" color="#CC0000">Access Denied</font></h1>
<p><font face="Arial,Helvetica">Please <a href="/403info.html">click here</a> for more information.</font></p>
</center>
</body>
</html>

Then make a second page that explains the 403 error in more detail. Very few robots will follow the link to that page. Call it "403info.html" so you can Allow access to it:

ErrorDocument 403 /403.html
#
Order Deny,Allow
Deny from 255.255.255.255
<FilesMatch "^403[^.]*\.html$">
Allow from all
</FilesMatch>
#
RewriteEngine on
RewriteCond %{HTTP_REFERER} http://([^.]+\.)*asite\.org [NC]
RewriteRule !^403[^.]*\.html$ - [F]

So you have two 403 files -- a very short one named "403.html" that provides only a link to more information, and a longer one named "403info.html" that explains the 403 error in case an innocent visitors gets a 403 because of a coding mistake or a server configuration error. Both files are "Allowed" in the mod_access and the mod_rewrite code above.

If you want to serve a 404, then you cannot use mod_access. Instead you must use mod_rewrite only, and rewrite the unwelcome requests to a non-existent URL-path on your server. This will cause a 404 if that is what you really want:

RewriteEngine on
RewriteCond %{REMOTE_ADDR} ^255\.255\.255\.255$ [OR]
RewriteCond %{HTTP_REFERER} http://([^.]+\.)*asite\.org [NC]
RewriteRule !^404\.html$ /this-URL-does-not-exist.hmtl [L]

The same problem exists as with 403: You must define a 404 error page, and make it as short as possible.

Jim

[edited by: jdMorgan at 12:10 am (utc) on May 31, 2007]

[Michel Samuel]

THANK YOU

That helped me a lot. I started getting into http server response codes and it just became a night mare.
________

This is what I did on the obligatore 403 file.

<% option explicit
response.Status="404 Page Not Found"
%>

I still get the 403 error but their systems also report a 404

--------

BUT...
This piece of code gives me a 500 erreur each time I include it with an http referer.

[([^.]+\.)*...]

___________

And when I remove it from the refer and go back to the origional now.
It gives me a 500 error but only for people going from the referer.

[edited by: Michel_Samuel at 7:24 am (utc) on May 31, 2007]

[Michel Samuel]

I now have it giving me 404 erreurs on everthing.
Not too certain how I managed to do that.

But it is doing it.

So I guess I am succesful. But this is disturbing.
I do not like not knowing how I managed to accomplish something.