Forum Moderators: phranque
For weeks I've been trying to figure out what is causing 500 errors. They seem intermittent and/or often occur immediately after I upload a new version of the htaccess file.
Think I've read almost every post ever made in the Apache forum, ;) studied the Apache directives, syntax and Library files. I just cannot find the problem.
The code below is an example of the type errors. Not asking anyone to fix it, just hoping it's an obvious error that I'm missing :( Notice the bolded "200s" that are in between 500s for the same IP and time stamp.
None of the IPs in the Get statements are even close to those which are actually in htaccess, none of the UAs are close to any with RewriteConditions in htaccess, and none of the files requested are, or ever have been, redirected.
I've cut&pasted every RewriteCond to make sure there are no typos
checked for broken pipes
made sure every line but the last in a conditional statement contains an [OR]
made sure there is a backward slash before every period in an IP
escaped any spaces, etc. in conditions.
made sure every RedirectPermanent is followed with a "/" before the file name and included the domain before the redirected "to" filename
"Deny from" never blocked any IPs so changed back to all rewrites.
Hosting service says "we don't support htaccess so can't help".
Now, I'm not at wit's end, 'cause it done ended middle of last month.
Since none of the IPs, UAs, or filenames logged in the error log have any connection to the listed error is there some other way to figure these errors out?
htaccess begins:
Options +FollowSymLinks
RewriteEngine On
RewriteCond %{HTTP_HOST} ^mydomain\.com
RewriteRule ^(.*) [mydomain.com...] [R=permanent,L]
#ErrorDocument 401 /err401.htm
#ErrorDocument 403 /err403.htm
ErrorDocument 404 /err404.htm
SetEnvIf User-Agent ^User ban
SetEnvIf User-Agent "compatible ;" ban
SetEnvIf User-Agent Charlotte ban
SetEnvIf Request_URI \.htaccess$ ban
SetEnvIf Request_URI \.htpasswd$ ban
<Files *>
Order Allow,Deny
Allow from all
Deny from env=ban
</Files>
folowed by RewriteConditions, hotlink code, several Mozilla rewriteconditions pulled from previous threads and RedirectPermanent's.
Example log file errors:
[Thu Sep 14 18:08:51 2006] [alert] [client 137.142.163.xx] /var/www/html/.htaccess: RewriteCond: bad argument line ''!
[Thu Sep 14 18:08:51 2006] [alert] [client 24.5.231.xx] /var/www/html/.htaccess: RewriteCond: bad argument line '%{HTTP_USER_'!
[Thu Sep 14 18:08:51 2006] [alert] [client 24.5.231.xx] /var/www/html/.htaccess: RewriteCond: bad argument line '%{HTTP_USER_'!
[Thu Sep 14 18:08:52 2006] [alert] [client 24.5.231.xx] /var/www/html/.htaccess: RedirectPermanent takes two arguments, a document to be redirected, then the destination URL
[Thu Sep 14 18:08:52 2006] [alert] [client 24.5.231.xx] /var/www/html/.htaccess: RedirectPermanent takes two arguments, a document to be redirected, then the destination URL
[Thu Sep 14 18:08:52 2006] [alert] [client 24.5.231.xx] /var/www/html/.htaccess: Invalid command 'RedirectPer', perhaps mis-spelled or defined by a module not included in the server configuration
[Thu Sep 14 18:08:52 2006] [alert] [client 24.5.231.xx] /var/www/html/.htaccess: Invalid command 'RedirectPer', perhaps mis-spelled or defined by a module not included in the server configuration
[Thu Sep 14 18:08:52 2006] [alert] [client 24.5.231.xx] /var/www/html/.htaccess: Invalid command 'Re', perhaps mis-spelled or defined by a module not included in the server configuration
[Thu Sep 14 18:08:52 2006] [alert] [client 24.5.231.xx] /var/www/html/.htaccess: Invalid command 'Re', perhaps mis-spelled or defined by a module not included in the server configuration
[Thu Sep 14 18:08:52 2006] [alert] [client 24.5.231.xx] /var/www/html/.htaccess: Invalid command 'Redi', perhaps mis-spelled or defined by a module not included in the server configuration
137.142.163.xx - - [14/Sep/2006:18:08:51 -0600] "GET /bsts/pnk.htm HTTP/1.1" 500 620 "http://www.mydomain.com/bsts/pur.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:51 -0600] "GET /images/sts/tny/amy.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:51 -0600] "GET /images/sts/tny/ame.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:51 -0600] "GET /images/sts/tny/amm.jpg HTTP/1.1" 200 1038 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:51 -0600] "GET /images/sts/tny/pet.jpg HTTP/1.1" 200 1080 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:51 -0600] "GET /images/sts/tny/gar.jpg HTTP/1.1" 200 1499 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:52 -0600] "GET /images/sts/tny/pic.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:52 -0600] "GET /images/sts/tny/gas.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:52 -0600] "GET /images/sts/tny/amm.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:52 -0600] "GET /images/sts/tny/pop.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:52 -0600] "GET /images/sts/tny/aaq.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:52 -0600] "GET /images/sts/tny/met.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
24.5.231.xx - - [14/Sep/2006:18:08:52 -0600] "GET /images/sts/tny/qqa.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/gsts/sts.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322)"
[Mon Sep 11 21:29:37 2006] [alert] [client 58.168.86.xx] /var/www/html/.htaccess: RedirectPermanent takes two arguments, a document to be redirected, then the destination URL
58.168.86.xx - - [11/Sep/2006:21:29:37 -0600] "GET /images/bsts/amy.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/bsts/feb.htm" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7"
[Sun Sep 3 16:08:31 2006] [alert] [client 24.29.198.xx] /var/www/html/.htaccess: Invalid command 'RewriteCo', perhaps mis-spelled or defined by a module not included in the server configuration
[Sun Sep 3 16:08:31 2006] [alert] [client 89.240.151.xx] /var/www/html/.htaccess: RewriteCond: bad argument line '%{'!
[Sun Sep 3 16:08:31 2006] [alert] [client 89.240.151.xx] /var/www/html/.htaccess: RewriteCond: bad argument line '%{'!
[Sun Sep 3 16:08:31 2006] [alert] [client 24.20.158.xx] /var/www/html/.htaccess: Invalid command 'RedirectPermane', perhaps mis-spelled or defined by a module not included in the server configuration
[Sun Sep 3 16:08:32 2006] [alert] [client 24.20.158.xx] /var/www/html/.htaccess: Redirect to non-URL
[Sun Sep 3 16:08:32 2006] [alert] [client 24.20.158.xx] /var/www/html/.htaccess: Redirect to non-URL
[Sun Sep 3 16:08:32 2006] [alert] [client 89.240.151.xx] /var/www/html/.htaccess: RedirectPermanent takes two arguments, a document to be redirected, then the destination URL
[Sun Sep 3 16:08:32 2006] [alert] [client 89.240.151.xx] /var/www/html/.htaccess: RedirectPermanent takes two arguments, a document to be redirected, then the destination URL
24.29.198.xx - - [03/Sep/2006:16:08:31 -0600] "GET /images/pen/chaty.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/gsts/cha.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
89.240.151.xx - - [03/Sep/2006:16:08:31 -0600] "GET /images/sts/tny/gar.jpg HTTP/1.1" 500 620 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; SV1; .NET CLR 2.0.50727)"
89.240.151.xx - - [03/Sep/2006:16:08:31 -0600] "GET /images/sts/tny/gar.jpg HTTP/1.1" 500 620 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; SV1; .NET CLR 2.0.50727)"
24.20.158.xx - - [03/Sep/2006:16:08:31 -0600] "GET /bsts/cit.htm HTTP/1.1" 200 26884 "http://www.mydomain.com/gsts/bsts.htm" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6"
89.240.151.xx - - [03/Sep/2006:16:08:31 -0600] "GET /images/sts/tny/pic.jpg HTTP/1.1" 304 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; SV1; .NET CLR 2.0.50727)"
24.20.158.xx - - [03/Sep/2006:16:08:31 -0600] "GET /images/err/palm-cws1ty.jpg HTTP/1.1" 200 2353 "http://www.mydomain.com/bsts/cit.htm" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6"
24.20.158.xx - - [03/Sep/2006:16:08:31 -0600] "GET /images/sts/yellow/citge.jpg HTTP/1.1" 200 17753 "http://www.mydomain.com/bsts/cit.htm" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6"
24.20.158.xx - - [03/Sep/2006:16:08:31 -0600] "GET /images/pen/druty.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/bsts/cit.htm" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6"
24.20.158.xx - - [03/Sep/2006:16:08:32 -0600] "GET /images/chriss/3ty.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/bsts/cit.htm" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6"
24.20.158.xx - - [03/Sep/2006:16:08:32 -0600] "GET /images/cus/pen/picrt.jpg HTTP/1.1" 500 620 "http://www.mydomain.com/bsts/cit.htm" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6"
89.240.151.xx - - [03/Sep/2006:16:08:32 -0600] "GET /images/sts/tny/pic.jpg HTTP/1.1" 500 620 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; SV1; .NET CLR 2.0.50727)"
89.240.151.xx - - [03/Sep/2006:16:08:32 -0600] "GET /images/sts/tny/amm.jpg HTTP/1.1" 500 620 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; SV1; .NET CLR 2.0.50727)"
[Sun Sep 3 13:54:20 2006] [alert] [client 24.118.180.xx] /var/www/html/.htaccess: RedirectPermanent takes two arguments, a document to be redirected, then the destination URL
24.118.180.xx- - [03/Sep/2006:13:54:20 -0600] "GET /bsts/sap.htm HTTP/1.1" 500 620 "http://www.mydomain.com/gsts/sap.htm" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
There is no reason for .htaccess to fail intermittently that does not have to do with some kind of server problem. If it fails, it should always consistently fail under the same set of circumstances, and not "fade in and out" as it appears to do in your log sample.
Then there's always the "What did you change recently?" question, and since you didn't mention anything, I assume that either you didn't change anything, or that this problem cropped up without any identifiable correlation to a specific change.
One thing I suggest doing (for everyone) is to upload .htaccess under a different filenname, and then rename it to .htaccess after the whole new file is on the server. You may be seeing the effects of an FTP delay in uploading the file, although the system should not access a new version of the file until it is complete and closed after writing... Still, to be safe, it's better to upload under a different name, then rename it once done. Maybe try that and see if it reduces or prevents this weird behaviour...
Jim
The log files at this host have always shown truncated directives like those shown and I've never been able to track down what the error really was.
As you suggested, I tried uploading the file under a diff name twice and renamed it, so far, no errors :)
Thanks for another tip! I've used the one you suggested a couple years ago - set the font to BIG - to find errors in htaccess and that's probably all that saved my eyes this past month.
I'll try uploading and renaming the file a couple more times tomorrow to double check.
Thanks again, Jim! :)
So, in fact, the requests which failed may have all happened while the .htaccess file was being uploaded and was incomplete, but a few successful requests might have been logged between the failed ones because they had actually been processed throught the URL-to-filename phase of the API before the .htaccess upload was started, but had then been deferred or delayed for some reason. So therefore, a few successful requests got logged in the middle of the failed ones, because they had started while the old .htaccess file was still in place, but finished after some of the the requests that failed while the .htaccess file was being uploaded and was incomplete and therefore invalid.
This is also the reason you may often see a request for an included CSS, favicon, or JS file logged before the log entry for the client requesting the "page" file itself, even though it's clear that the client is a first-time visitor, and could not have "known" the names of the included files without first fetching and parsing at least part of the page. What you see in the logs should therefore be an impossibility -- and it is if you interpret the log as indicating the order of HTTP request arrivals. But this happens much more often if the include files are short and the page file is long, and offers circumstantial evidence that the log shows order of HTTP request completion instead.
The only time this is of concern is when dealing with log entries that are time-stamped within a very few seconds of each other. It's unlikely that a first group of requests logged more than three seconds before a second group of requests will contain any requests that arrived after any one in the second group, unless the server is quite overloaded or the site uses very slow scripts or a badly-organized database.
Jim
Since I've been renaming the htaccess after uploading there have been no more of those pesky truncated directives. File seems to be working as it should, even the DENY FROM directives that I added back in. :) Still testing though.
