html linked JS and CSS files loading very slow

JS relative links load slow, but if you load directly through URL no issue

[rodbland]

We are having an almost identical issue to this one posted earlier in the year:

[webmasterworld.com...]

Webserver details: Apache/2.0.63 Unix mod_ssl/2.0.63 OpenSSL/0.9.8e-fips-rhel5 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635

Our cerberus helpdesk application loads several js and css files in the header. Up until a week ago this application was always snappy, but now we are seeing several clients experiencing very slow load times for what are only small components.

The yslow output from a client machine with this problem which exists on the same subnet as the webserver is here:

[ramcity.com.au ] Note the 15 second load time for the js file, and over 24 seconds for the css file.

Either of these files can be loaded direct in the browser in sub 1-second time.

It doesn't appear to be related to any particular client browser, OS, or network location. We have some staff with no problems, and some which see slowdown at the beginning of the day, then ok after lunch, and some the reverse.

Someone on the cerberus forums suggested they have seen something similar with drupal related to a maximum number of files open in Apache.

This server is hosted with Amazon but we haven't ponied up for a subscription to ask them yet. Hoping perhaps someone here might have some ideas?

cheers, Rod

[jdMorgan]

> Up until a week ago

This is the key. What changed one week ago?

Also look at the error logs for Apache and cerberus -- and for mod_bwlimited (if it generates one). Error log files often tell you exactly what is wrong, or give a very strong clue.

You've got a whole lot of trails to follow...

Jim

[rodbland]

The Apache error_log has a reference to a missing deflate reference, which we are pretty sure is mod_gzip not being compiled in. Not sure if this has anything to do with our issue, but we're going to sort that out and see if it helps. Apart from that, no other errors are noted when this issue occurs.

I'll post back again once the recompile is done.

[jdMorgan]

The Apache access log will have that info for every request you sent to your server with Yslow...

Jim

[rodbland]

the compress issue is sorted, but sadly that didn't help us.

We don't have anything showing in the error_log other than the odd user with a password failure.

Here is an example from the acess log (only 4 components on this page, less than 20KB, but still took about 17 seconds to load)

aws2.ramcity.net - rod [01/Nov/2010:11:35:43 +1100] "GET /cerb5/display/XEN-95635-591 HTTP/1.1" 200 5048 "http://servicedesk.ramcity.net/cerb5/explore/facac2112a2d2be09292de650faf4191/2" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.1.14) Gecko/20101001 Firefox/3.5.14 (.NET CLR 3.5.30729)"

**(above is initial page request)**

aws2.ramcity.net - rod [01/Nov/2010:11:35:43 +1100] "GET /cerb5/resource/cerberusweb.core/css/cerberus.css?v=2010092001 HTTP/1.1" 200 3437 "http://servicedesk.ramcity.net/cerb5/display/XEN-95635-591" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.1.14) Gecko/20101001 Firefox/3.5.14 (.NET CLR 3.5.30729)"

** (above is the first css file that's needed, downloaded in the same second as the initial page request) **


aws2.ramcity.net - rod [01/Nov/2010:11:35:43 +1100] "GET /cerb5/resource/devblocks.core/css/jquery-ui.css?v=2010092001 HTTP/1.1" 200 6027 "http://servicedesk.ramcity.net/cerb5/display/XEN-95635-591" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.1.14) Gecko/20101001 Firefox/3.5.14 (.NET CLR 3.5.30729)"

** (above is the second css file, note the 16 second gap between when this file is requested, and the next request served below. For 6027 bytes?) **


aws2.ramcity.net - rod [01/Nov/2010:11:35:59 +1100] "GET /cerb5/ajax.php?c=display&a=showConversation&ticket_id=49770 HTTP/1.1" 200 3664 "http://servicedesk.ramcity.net/cerb5/display/XEN-95635-591" "Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.1.14) Gecko/20101001 Firefox/3.5.14 (.NET CLR 3.5.30729)"

** (Above, at last the page is rendered, 16 seconds from the initial page request) **

I have to admit, I'm a bit out of my depth trying to troubleshoot this. Hopefully something stands out here to someone else?

[rodbland]

Ok I figured this one out.

Once I started looking at the page output with [fiddler2.com ] I could see messages like this:

Fiddler.Network.ProtocolViolation - [#5] Content-Length mismatch: Response Header claimed 7162 bytes, but server sent 3701 bytes.

It seems that whatever was doing the compression wasn't doing the job properly, so I went through and commented out all mod_deflate compression in /cerb5/.htaccess

Firebug/yslow still showed the pages were being compressed so I checked php.ini and also commented out zlib compression like so:

# zlib.output_compression = On

Once that was done, the problem went away. The browser was pausing for 15 seconds each time it got a mismatched header response while it waited for the extra data it was never going to get. I guess it must be a bug in php or something in cerberus itself.

The only issue now is that I can't get compression working using the usual methods via .htacess - I think perhaps because cerberus uses those special version characters at the end of each .css and .js file that is loaded in the header. Has anyone successfully got gzip compression working where everything is compressed with some exceptions such as images and gzipped content, and would mind sharing how they set that up?

cheers, Rod

[rodbland]

I finally did figure this one out, and we have now compression happily working, even for js and css files with version parameters.

Oddly enough, the issue appeared to be related to either the Expires header not being set, or because ETag was set. I'm not sure which, but the solution was to unset ETag and set proper long term expiry on the relevant content as follows: (put this in your .htaccess file or apache config file (/usr/local/apache/conf/httpd.conf on my server)

#####################################################
# CONFIGURE media caching
#
Header unset ETag
FileETag None
<FilesMatch "(?i)^.*\.(ico|flv|jpg|jpeg|png|gif|js|css|cssimage)$">
Header unset Last-Modified
ExpiresActive On
ExpiresDefault "access plus 2 months"
Header set Cache-Control "public, no-transform"
</FilesMatch>
#
#####################################################

Once I implemented the above, then mod_deflate started working on all the css and js content as well. Here is the code I use for mod_deflate, also in .htacess, or the apache conf file:

<ifmodule mod_deflate.c>
SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ \
 no-gzip dont-vary
SetEnvIfNoCase Request_URI \
 \.(?:exe|t?gz|zip|bz2|sit|rar)$ \
 no-gzip dont-vary
SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary
</IfModule>

Hope this whole post is of some use to someone else in the future.

cheers, Rod

[jdMorgan]

Thanks for posting your solution!

Jim