mod_deflate

getting it going on my server

[mipapage]

Hello,

I'm trying to use mod_deflate on my server. I've read around a lot, tried this and that, and not had any luck (I think).

I've enabled the module...

LoadModule deflate_module modules/mod_deflate.so

and tried some of the examples provided here:
[httpd.apache.org...]

with no luck, or so it seems...

I'm using

LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' deflate 
CustomLog logs/deflate.log deflate

To see if things are working, taking the data from

DeflateFilterNote Input instream 
DeflateFilterNote Output outstream 
DeflateFilterNote Ratio ratio

as indicated at the above link.

Now I'm not sure if deflate is working, and the logs aren't, or if it's all out to lunch.

Any ideas?

[mipapage]

Okay,

This is what I have in my httpd.conf for the log files:

<IfModule mod_deflate.c> 
DeflateFilterNote Input instream 
DeflateFilterNote Output outstream 
DeflateFilterNote Ratio ratio 
LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%) "%{User-agent}i"' deflate 
CustomLog logs/deflate_log deflate env=!trash 
 </IfModule>

The log file is being written, so I can assume that the module is loaded.

However, the results of the log file look like this:

"GET /somepath/somefile.html HTTP/1.1" -/- (-%) "insertuseragenthere"

So for whatever reason, it seems that things aren't compressing...

[uncle_bob]

I'm using a much simpler conf

<IfModule mod_deflate.c>
AddOutputFilterByType DEFLATE text/html
SetInputFilter DEFLATE
</IfModule>

This writes the deflated size to the normal access log if deflated, or the ordinary size if not. Does not show the before and after sizes as I already know the before size and just want to know if it was deflated or not.

[mipapage]

Thanks uncle_bob,

I gave yours a try and still I get the uncompressed value.

[uncle_bob]

Couple of things to check,

make sure you are using %b for the size in the custom log format, and secondly, check the browser is supporting compressed files. I use the http headers plugin in firebird to check exactly was is being sent and received header wise. Very enlightening. From my log files, I see that IE6 doesn't always get the compressed file.

[mipapage]

uncle_bob,

Thanks again. I tried the changes and tried out the extension in firebird - pretty cool stuff.

Anyway, no luck. I have tried ie6, opera 7.11, and Moz, as well as firebird.

For some reason it's not applying the filter. Online doc's are pretty limited it seems, and the only articles out there seem to be the same one in many different places (the one by stephen from sitepoint(?)).

One guy mentioned that it worked for him depending on which modules he had loaded! I'm not that desperate... yet.

Yesterday I thought to myself 'phew, glad I have apache 2 installed and not 1.x. Now I don't have to try and make mod_gzip work.' Clearly the Apache gods have a sense of humour...

[mipapage]

Okay,

tried this out on our live server, and still no luck.

I placed this code...

LoadModule deflate_module modules/mod_deflate.so 
AddOutputFilterByType DEFLATE text/html

...right after the 'loadmodule' section.

Interestingly, this site - ht*p://leknor.com/code/gziped.php - will check live documents and report if they are gzipped or not. This site reports that mod_deflate is working on our server, and indeed it shows in the access logs as:

***.**.**.*** - - [29/Dec/2003:17:01:55  
-0800] "GET /a_file.html HTTP/1.0" 200 1653

This (1653) is the zipped size, however, this is the only time I've seen it work - I've never been able to record a zipped file size with any real browsers (IE, Moz, Opera, Firebird etc.).

Strange too that it reports as HTTP/1.0, when gzipped encoding is only supposed to work for HTTP/1.1.

Anyone out there have any ideas?

[uncle_bob]

Are you accessing your website through a proxy server? Using the HTTP Headers plugin in firebird, do you see the following line in the HTTP request?

Accept-Encoding: gzip,deflate

Not really sure what could cause it not to work in every browser you've tried. I do know that my log files show that it doesn't always return gzipped files to everyone. So clearly not all browsers (or user agents) support it.

[mipapage]

Fwiw, here's what it tells me:

GET /a_file.html HTTP/1.1
Host: 80.28.176.242
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko/20031007 Firebird/0.7
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2,*/*;q=0.1
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache

HTTP/1.x 200 OK
Transfer-Encoding: chunked
Date: Tue, 30 Dec 2003 14:02:15 GMT
Server: Apache/2.0.47 (Win32) PHP/4.3.2
Last-Modified: Fri, 28 Nov 2003 16:08:25 GMT
Accept-Ranges: bytes
~~~~~~~~~~~~~~: ~~~~
Vary: Accept-Encoding
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=ISO-8859-1

I find it weird about the 'ht*p://leknor.com/code/gziped.php' link as well. It tells me that gzip/deflate is working, while I am pretty sure it isn't.

One of the 'blogs I read while trying to get this going recommended testing your setup there; good idea, but if the results are unreliable...?

Thanks again for the help.

[mipapage]

Okay,

seems that it must be working. I've tested here:

ht*p://www.xcache.com

and they returned a report saying that, yes, compression is on. It was even reflected in my logs. It would seem then that for whatever reason our server is not delivering compressed files to me, and could explain why the log details werent recorded.

xcache also shows as 'HTTP/1.1' in the logs.

As to why the browsers I am using aren't getting the zipped content is beyond me, it'll be interesting to see how many users get zipped content versus those that don't.

[jdMorgan]

mipapage,

Please keep us posted...

A previous thread, combined with yours, has managed to get me very interested in trying this out, and I'm following your project intently.

Thanks,
Jim

[uncle_bob]

Mipapage,

I just compared you http headers to mine, and we are using the same browser (exactly the same request headers) but the response headers are quite different. The only server difference is I'm running Apache 2.0.48 not 2.0.47, on win32 with php. What I get is

HTTP/1.x 200 OK
Date: Tue, 30 Dec 2003 18:29:01 GMT
Server: Apache
Last-Modified: Tue, 02 Sep 2003 16:43:03 GMT
Etag: "80-148-71ce840b"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Cache-Control: max-age=600
Expires: Tue, 30 Dec 2003 18:39:01 GMT
Content-Length: 248
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=ISO-8859-1

[mipapage]

Fwiw, here's the headers that xcache returned to me (these are the folks said it was working):

HTTP/1.1 200 OK 
Date: Tue, 30 Dec 2003 14:29:17 GMT 
Server: Apache/2.0.47 (Win32) PHP/4.3.2 
Last-Modified: Fri, 28 Nov 2003 16:08:25 GMT 
ETag: "12c30-d97-1a20dea4" 
Accept-Ranges: bytes 
Vary: Accept-Encoding 
Content-Encoding: gzip 
Content-Length: 1653 
Content-Type: text/html; charset=ISO-8859-1

---

jdmorgan,

I won't let this die, I've asked my hosting company to activate it for me (other servers), and I'd like to be able to help if they give me any grief!

I should check what version of apache they are running...

[uncle_bob]

Those xcache headers match what I'd expect. Mine have extra keep-alive bits thats all. I'm surprised when you access the webserver from firebird you don't get the same headers.

The version of apache you are running is in your headers (2.0.47)

update - woo whoo ! Just gained a level to 'Junior Member' from 'New User' . Where do I roll for my extra hit points?

[mipapage]

Congrats on te upgrade uncle_bob...

Get this, here is the request header from xcache:

GET /a_file.html HTTP/1.1 
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705)  
Host: 80.28.***.*** 
Accept: */* 
Accept-Language: en-us 
Accept-Encoding: gzip,deflate 
RequestType: 1st

I.E. six, no? Why does it feel like my head is banging against a hard object? ;-]

I may go and upgrade to .48 ...

[mipapage]

Okay, I need a little help with this one:

This page:
ht*p://www.performancecorps.org/browser_check.php

I went there and tried all of my broswers and they all returned

"1. Your browser does not support GZIP Encoded pages. GZIP Encoding compresses HTML files so that the same data can be transferred in fewer bytes."

So, for whatever reason (my ISP?) my browsers aren't sending through the correct headers (in msg#9 it shows that firebird does, but that was all local).

The next step (if anyone is interested): I'll sticky someone an URL to my server, and we'll see what happens when they hit it?

---

And an extra 'Happy New Year' if someone can explain why my browsers are all... gzip challenged..

[mipapage]

And an extra 'Happy New Year' if someone can explain why my browsers are all... gzip challenged..

Solved!

---

Okay, a little wake up call from some outside help clued me in to trying to download a page with my firewall off.

Seems that with my firewall on, I can't download gzip'ed content from my server, with it off, I can.

Now that begs to question - can I download anything gzip'ed with my firewall on?

<off I go...>

---

<added>

uncle_bob - what if those IE users that you have seen were perhaps behind firewalls?

Would you have an idea of how many pages come off of your server compressed vs. non-compressed?

[uncle_bob]

Would you have an idea of how many pages come off of your server compressed vs. non-compressed?

At a quick guess I'd say 8 or 9 percent of the pages delivered are not gzipped.

[mipapage]

Thanks.

Weird stuff - Google comes down compressed whether my firewall is up or down, according to firebird and the header tool**

But off the server in the next room, no luck!

We'll, I'm going to give it a shot live as soon as our hosts get it set up (I'm told first thing in the new year).

---

**Also according to running this php file on my server:

<?php  
print_r ($_SERVER); 
?>

When the firewall is on, "[HTTP_ACCEPT_ENCODING] => gzip, deflate " shows up as "[HTTP________] => ~~~~:~~~~~~~~~~~~~~~~~~~"