old solution; new problem

custom error document not found if error was because of rewrite

[x170]

Hi

I found this solution:

[webmasterworld.com...]

Very helpfull; and now I have unlimited number of subdomains using only one rule in the future.

Very useful and thank you for that.

However I have a problem now with my 404 page.

I have set up custom errorpages and the 404 works like a charm when it comes to a file that doesn't exist:

[mydomain.com...]

gives my 404 page. So I may assume that it works fine.

If I use

[DIR.mydomain.com...]

and /subs/DIR/ does exist; everything goes well. Just as I needed.

However (and there is my problem I could not fix)
if someone does this:

[DIR2.mydomain.com...]

and /subs/DIR2 does NOT exist; I get this:

--
Not Found

The requested URL /subs/test2/ was not found on this server.

Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
--

Why is that? He seems not to be able to find my 404 page.

Thanks for any help with this!

Matthijs

[Caterham]

Why is that? He seems not to be able to find my 404 page.

As you said

and /subs/DIR2 does NOT exist

If the resource is within the scope of your rules and is not excluded, e.g., with a condition it'll match your rule, too.

[x170]

Hi Catergam, thanks for helping!

I have in my httpd.conf this:

ErrorDocument 400 /error/400.php
ErrorDocument 401 /error/401.php
ErrorDocument 403 /error/403.php
ErrorDocument 404 /error/404.php
ErrorDocument 500 /error/500.php

So what you're saying is:

Apache is trying to get the 404 from

[DIR2.mydomain.com...] ?

I thought he would still use my document root, e.g.

/var/www/mydomain/html/error/404.php

Anyway; any ideas how to fix this?

So this is what I have:

# Internally rewrite <subdomain>.example.com/<URLpath> to example.com/subs/<subdomain/<URLpath>
rewriteCond $1 !^subs/
rewriteCond %{HTTP_HOST} !^mydomain\.com
rewriteCond %{HTTP_HOST} ^([^.]+)\.mydomain\.com
rewriteRule (.*) /subs/%1/$1 [L]

What I would like is that if the directory doesn't exist he will give the normal 404 page.

I tried
rewriteCond /subs/%1 !-d

but that didn't work as I hoped.

Thnx!

[jdMorgan]

You're missing the server document root filepath in your "exists" check, and the sense of the check is inverted. Try:

# Internally rewrite <subdomain>.example.com/<URLpath> to example.com/subs/<subdomain/<URLpath>
# if the subdomain's subdirectory exists
RewriteCond $1 !^subs/
RewriteCond %{HTTP_HOST} !^example\.com
RewriteCond %{HTTP_HOST} ^([^.]+)\.example\.com
RewriteCond %{DOCUMENT_ROOT}/subs/%1/ -d
RewriteRule (.*) /subs/%1/$1 [L]

Jim

[x170]

Thanks Jim

This is working: it will only rewrite if the sub directory exist.

However I still would like to produce the 404 if the sub directory does not exist.

Right now:

[DIR2.mydomain.com...] will just be displayed. How to produce a 404 for this?

something like this?

RewriteCond $1 !^subs/
RewriteCond %{HTTP_HOST} !^example\.com
RewriteCond %{HTTP_HOST} ^([^.]+)\.example\.com
RewriteCond %{DOCUMENT_ROOT}/subs/%1/ !-d
RewriteRule (.*) %{DOCUMENT_ROOT}/error/404.php [L]

(haven't tried it yet ;-) )

And to be honest; I could figure out how to generate a 404 with rewriterule. Is that even possible?

thanks!

[jdMorgan]

The 404 *will* be produced by default -- You don't need to add code to do it.

Simply define your 404 page using the ErrorDocument directive

ErrorDocument 404 /error/404.php

See Apache core ErrorDocument documentation.

If this does not work, then you must have some other code interfering with the server's basic error-handling functions; Fix that code to eliminate the interference.

Jim

[x170]

ah I know.

But in my case:

- the directory isn't there
- the rewrite rule ignores this one
- so [DIR2.mydomain.com...] will be shown as it was the normal website -> it will show the site as if it was www.mydomain.com

I have a DNS A record * and because of that I can have all these subdomains. Valid subdomains (the directory exist) will be taken care of as intended; but I still would like to show a 404 if the subdomain is not valid.

any thoughts?

[x170]

ah

it works! I get my own 404 error document with:

RewriteCond $1 !^subs/
RewriteCond %{HTTP_HOST} !^example\.com
RewriteCond %{HTTP_HOST} ^([^.]+)\.example\.com
RewriteCond %{DOCUMENT_ROOT}/subs/%1/ !-d
RewriteRule (.*) %{DOCUMENT_ROOT}/error/404.php [L]

I will use it for now (probably forever ;-) but I can't help thinking this is not the optimal way to do this

[jdMorgan]

Do not use %{DOCUMENT_ROOT} for the substitution URL-path -- that is wrong.

Rewriting to your error page is not the correct solution anyway. You will get a 200-OK response instead of a 404, and this will confuse search engines and likely affect your sites' rankings in search.

Try this instead:

RewriteCond $1 !^this_path_does_not_exist
RewriteCond %{HTTP_HOST} ^([^.]+)\.example\.com
RewriteCond %{DOCUMENT_ROOT}/subs/%1/ !-d
RewriteRule (.*) /this_path_does_not_exist [L]

By rewriting the request to a path that does not exist, you will force a correct 404 response. You can use any non-existent path you like; The one I show is just an example.

Jim

[x170]

Okay will try that.

Btw I looked closer to the second part of the original topic:

# Externally redirect client requests for example.com/subs/<subdomain>/<URLpath> to <subdomain>.example.com/<URLpath>
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /subs/
RewriteRule ^subs/([^/]+)/(.*)$ [$1.example.com...] [R=301,L]

That last line should be:

RewriteRule ^subs/([^/]+)/(.*)$ [$1.example.com...] [R=301,L]

[x170]

hmmm

when I do that I am back to my original problem:

--
Not Found

The requested URL /this_path_does_not_exist was not found on this server.

Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
--

the error document cannot be found.

[x170]

How about if I use:

RewriteRule (.*) http://example.com/this_path_does_not_exist [L]

[x170]

ok that works

one downside is that I end up with:

http://example.com/this_path_does_not_exist

instead of the original request.

[g1smd]

If you include a domain name in a RewriteRule then that rule WILL serve a 302 redirect to the new URL (even without an R flag included).

Omit the domain name and the Rule will be an internal rewrite - and that is what you need here.

Again, as coded, your 404 page is currently being served with a '302 Found' HTTP response code.

[x170]

Hi g1smd,

but if I leave it out; I don't get my 404 error document :(

[g1smd]

Flush your browser cache and try again.

Including the domain name will mean your server does NOT send a 404 status code in the HTTP header.

If there is no 404 status code in the HTTP header, then it isn't (by definition) a 404 page.

[x170]

I get what you mean: don't include the domain name or you won't get a 404.

I want a 404; but why doesn't he show my custom error document? And that is what I want <- I want my custom error document.

Instead I get this: Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.

[g1smd]

Ah, you haven't told the server where the file to serve when a URL is invalid is actually located.

You'll need:

ErrorDocument /errordocs/error404.php

- or wherever it is, and whatever it is called, on your server.

Note that there must be no domain name mentioned in this directive either.

[x170]

But I did that already; it is in the httpd.conf:

ErrorDocument 400 /error/400.php
ErrorDocument 401 /error/401.php
ErrorDocument 403 /error/403.php
ErrorDocument 404 /error/404.php
ErrorDocument 500 /error/500.php

because whenever I do: [mydomain.com...] it will show the correct errordocument ; so it is working

maybe I should move the ErrorDocument statement to my htaccess and include the serverpath? or?

I am totally lost :)

[jdMorgan]

It's likely that the ErrorDocument path defined in httpd.conf is wrong.

Add the URL-path you've defined in the ErrorDocument directive to the filepath you've defined as DocumentRoot. Is that the correct location in the filesystem where the error page can be found?

When you get a 'failure' to serve the correct ErrorDocument by requesting a non-existent subdomain, what do you see in the server *error* log file?

Jim

[x170]

Hmmm I have multiple domains running on my server. And most confusing is that it does work for 'normal' files that cannot be found.

this is what is in httpd.conf
DocumentRoot "/var/www/html"

Then almost on the bottom:

ErrorDocument 400 /error/400.php
ErrorDocument 401 /error/401.php
ErrorDocument 403 /error/403.php
ErrorDocument 404 /error/404.php
ErrorDocument 500 /error/500.php

and below that are the includes per domain.

Then for this particular domain there is :

<VirtualHost 123.123.45.56:80>
DocumentRoot /home/USER/domains/example.com/public_html

... and more but no ErrorDocument stuff

</VirtualHost>

and that is the correct location of the root.

I will dig up the error log

[x170]

okay if I open a page that does not exist:

(normal log)
[14/Mar/2009:01:34:39 +0100] "GET /normalpage HTTP/1.1" 404 19 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.5) Gecko/2008120121 Firefox/3.0.5"

(error log)
[Sat Mar 14 01:34:39 2009] [error] [client 70.78.227.1] File does not exist: /home/mydomain/domains/mydomain.com/public_html/normalpage

--

now I will go to a non existing sub domain ( <- so the directory is not there )

[notthere.domain.com...]

gives me in the browser:

-
Not Found

The requested URL /this_path_does_not_exist was not found on this server.

Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
-

Hmmm weird: I get 5 (!) of these in the error log

[Sat Mar 14 01:40:12 2009] [error] [client 70.78.227.1] File does not exist: /home/mydomain/domains/mydomain.com/public_html/this_path_does_not_exist

And the normal log gives me:

[14/Mar/2009:01:40:12 +0100] "GET / HTTP/1.1" 404 - "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.5) Gecko/2008120121 Firefox/3.0.5"
[14/Mar/2009:01:40:13 +0100] "GET /favicon.ico HTTP/1.1" 404 - "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.5) Gecko/2008120121 Firefox/3.0.5"
[14/Mar/2009:01:40:16 +0100] "GET /favicon.ico HTTP/1.1" 404 - "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.5) Gecko/2008120121 Firefox/3.0.5"

I noticed that the 404 page is not 19bytes as with the 'normal' page.

...?

[Caterham]

You're on apache 2.2+? Don't make it too complex.

<VirtualHost 123.123.45.56:80>
DocumentRoot /home/USER/domains/example.com/public_html
ServerName ....
ServerAlias ....
#
RewriteEngine on
RewriteCond $1 !^/subs/
RewriteCond %{HTTP_HOST} ^([^.]+)\.example\.com
RewriteCond %{DOCUMENT_ROOT}/subs/%1/ -d
RewriteRule ^(.*) /subs/%1/$1 [L] 
#
RewriteCond %{HTTP_HOST} ^([^.]+)\.example\.com
# the dir does not exist
RewriteCond %{DOCUMENT_ROOT}/subs/%1/ !-d
# issue 404 rather serving files from the main domain
RewriteRule !^/error/ - [R=404]
</VirtualHost> 

[edited by: Caterham at 12:45 am (utc) on Mar. 14, 2009]

[x170]

too bad.. Apache/2.0.59 (Unix)

[Caterham]

In that case I'd drop apache's 404 generation, rewrite to the php script directly and issue the status code 404 via php's header() function.

i.e. change th line

RewriteRule !^/error/ - [R=404] into

RewriteRule !^/error/ /error/404.php [L] 

and make sure your php script starts with

header("HTTP/1.0 404 Not Found");

if php is invoked via mod_php;

or

header("Status: 404 Not Found"); 

if you run php via CGI/FastCGI etc.

[x170]

should I upgrade ?

I was looking into that.

btw thnx for your time! :)

[x170]

Okay I upgraded to v2.2.11

If I open a non-existing subdomain I see:

--
Not Found

The requested URL / was not found on this server.

Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.

--

the error log gives me:

Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace. ( 5 times )

and this is what I use:

RewriteCond %{HTTP_HOST} ^([^.]+)\.example\.com
# the dir does not exist
RewriteCond %{DOCUMENT_ROOT}/subs/%1/ !-d
# issue 404 rather serving files from the main domain
RewriteRule !^/error/ - [R=404]

the normal log gives me:

[14/Mar/2009:06:39:05 +0100] "GET / HTTP/1.1" 500 566 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.5) Gecko/2008120121 Firefox/3.0.5"

--

I was wondering... I have the rewrite stuff in my .htaccess. You put it in the httpd conf. Does that make a difference?

Thanks people!

[g1smd]

It's more efficient in httpd.conf because the settings are compiled in, but you will need to restart the server software every time you make a change to the configuration.

Note too, that you will need to specify a leading / on any RewriteRule patterns, whereas you omit them when used in .htaccess.

[Caterham]

You put it in the httpd conf. Does that make a difference?

The directory context is more complex due to its architecture. If you have access to the httpd.conf don't make the processing more complex than necessary.

When (not) to use .htaccess files [httpd.apache.org]