homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Code, Content, and Presentation / HTML
Forum Library, Charter, Moderators: incrediBILL

HTML Forum

This 37 message thread spans 2 pages: 37 ( [1] 2 > >     
Cutting Bandwidth Needs

 12:54 pm on Dec 3, 2002 (gmt 0)

Here's a good example of WebmasterWorld synchronicity - I ran across the "build for slow guys topic the same evening I got a "we're gonna bill you for extra bandwidth" notice from one of my hosts. One site I manage is a victim of its own success, generating 25 GB of bandwidth usage last month.

A lot of the bandwidth comes from a discussion forum. These pages aren't graphic - at most, they load a gif banner. They are, of course, text-heavy. So, I'm looking for ways to cut down on bandwidth usage unrelated to images.

One thing I'm going to do is some archiving to cut down on "big" pages, particularly those that get loaded frequently.

Any other suggestions for cutting bandwidth, particularly in a mostly text-based environment?



 1:11 pm on Dec 3, 2002 (gmt 0)

Putting as much as you can into external CSS and JS files that would get chached on the users machine - saving on multiple requests and used bandwidth for repeated elements.

brotherhood of LAN

 1:38 pm on Dec 3, 2002 (gmt 0)

I think Brett has changed this over the year ;) but delete all your tabs and spaces between tags, for sure.

There used to be a few good examples of big sites with fat indents that must have cost a bit $ in bandwidth.

The home page of WW hardly has any line breaks or indents, but you still save almost 1%....i.e. 250Meg? :)

When I was playing around with spidering and indexing pages, a more "assorted" set of pages were getting cut down by about 10% by removing line breaks, double spaces etc.

Probably just a good a saving for most people as external calls to jscript/css, caching etc


 1:57 pm on Dec 3, 2002 (gmt 0)

Doing a Google search for html compression tool can point you towards some very handy apps that will get rid of any unwanted space.

As brotherhood_of_LAN said, this can cut down a surprising amount of your file size.


 1:48 pm on Dec 4, 2002 (gmt 0)

Thanks for the suggestions. Since the pages are auto-generated by the discussion software, I guess my main hope is to go through the templates and see if I can hammer any room out of them. The compression software sounds interesting, although unless I have a solution that works "on the fly" the compression effect will be lost as soon as a page is regenerated. Unlike WebmasterWorld, my forum pages are static html that are recreated if someone adds to that page. Topics pages probably account for a lot of the bandwidth, and unfortunately they are regenerated frequently.


 7:17 pm on Dec 4, 2002 (gmt 0)

  • Reduce the number of posts a page displays from say 20 to 10?
  • Remove comments and copyright notices from the scripts?
  • Modify <strong> to <b>? <em> to <i>?
  • Change absolute links to relative?
  • EliteWeb

     7:18 pm on Dec 4, 2002 (gmt 0)

    Apache Server? Have administration use mod_gzip to help ease bandwidth - saved mine 1/2 :P


     9:01 pm on Dec 4, 2002 (gmt 0)

    Eliteweb, will the mod_gzip work in a shared environment? Does it place a bigger workload on the server? Thanks.


     9:39 pm on Dec 4, 2002 (gmt 0)

    Check your images, client-side scripts, and stylesheets to make sure they are cacheable [ircache.net].



     10:05 pm on Dec 4, 2002 (gmt 0)

    mod_gzip [sourceforge.net] does not increase server memory usage at all that i can notice. and tests show that mod_gzip'd pages actually transmit quicker to the clients too. Any modern browser can decompress gzip pages on the fly without any slowness noted. worth a look at. i think every server should use it anyways :) however bandwidth is what they charge you for :P


     10:45 pm on Dec 5, 2002 (gmt 0)

    usefule thread. (starts to worry about bandwidth - sees guilty face in mirror)


     11:01 pm on Dec 5, 2002 (gmt 0)

    One has to wonder about the overhead trying to compress on-the-fly for zip files and JPEGs. You bandwidth could actually increase if your site is predominantly compressed files or JPEGs.


     11:50 pm on Dec 5, 2002 (gmt 0)

    Good point, sun818. In this case, I think text and HTML are my primary culprits, though.


     2:35 pm on Dec 7, 2002 (gmt 0)

    I have to agree on goin external for css/js for high page-views-per-user rates. I think there needs to be a qualifer to that, because per user page views can vary widely.

    I have been adjusting my little rule of thumb when it is advisable to go external. I think it is a mix of 3 issues with fuzzy gray lines inbetween them.

    1) Will going external reduce the html size by atleast 25%? If so, go external.
    2) Will going external actually reduce the html-css-js total in size? If not, stay on page.
    3) page views per visitor greater than 5? If so, go external.
    4) More than 50% of your daily uniques from search engines? Keep the css/js on page.
    5) How many pages does a user have to view to see a return on bandwidth savings by going external?
    6) How many users will see a negative download impact by going external? If it's more than 10%, you may want to stay on-the-page.
    7) How many visitors will you lose if you go off-the-page with CSS?

    I think 5-6 are the important ones. You have to judge your cutoff point where the return bandwidth savings is worth it, or is going to hurt you.


     5:53 pm on Dec 7, 2002 (gmt 0)

    This discussion [webmasterworld.com ] may interest you. It's about people with slow connections (56K modems) but there are some comments about software compression and reducing the size of HTML pages.

    Personally, if I were you:
    - I'd look for a software compression (gmod-zip for Apache or XCompress for IIS). You can obtain a gain up to 80% (at least this is what I gained in my ASP pages using software compression, 60% if we considered also all non HTML/ASP files).
    - I'd link externally all CSS and JS (IMPORTANT: and make them cacheable, because if not all users would download it several times among the same session)
    - I would make images cacheable
    - I'd try to make images smaller
    - I'd delete all useless HTML code: <strong>; <em>; images ALT, width and height (where possible)
    - I'd look the logs to see if there's any "bad boy" bot which is stealing bandwidth from you


     6:06 pm on Dec 7, 2002 (gmt 0)

    >More than 50% of your daily uniques from search engines? Keep the css/js on page.


    I get the other 6 points, but can you please expand a bit on this one?




     7:34 pm on Dec 7, 2002 (gmt 0)

    I run a couple discussion boards myself, my advices would be:

    - reduce number of displayed posts per page
    The viewtopic-page is drawing by far the most traffic accross our boards, so even little reduction will save a lot over all.

    - mod_gzip (on apache)
    Ask your hoster for this. I talked with mine, and he happily installed it. Some will argue that it takes too much processing time, tell them to just try it for a day, the benefits usually outweigh the drawback by far. The CPU usage is not as heavy as some people think.

    Otherwise gzip your pages in your scripts. This is not as efficient as mod_gzip, however you are not dependant on your hoster to install it. For PHP look here for example: [leknor.com...] (class.gzip_encode.php). Some BBs have implemented this sort of compression and have an option to turn it on/off in the configuration (www.phpBB.com for example)

    Let this test: [leknor.com...] make a guess on how much you could save with gzip compression. It is highly effective on text (!)


     8:13 pm on Dec 7, 2002 (gmt 0)

    Another thing you could do, if you have a conventional table-layout (a lot of BBs have a whole lot of nested tables) is you could switch to a CSS-layout. See what it did for wired: [stopdesign.com...]

    You might also ask the dev/support team of the BB if there is sth. like that available. Maybe someone has done it already.


     10:02 pm on Dec 7, 2002 (gmt 0)

    Search in google for a tool called "html schrinker". It can reduce you code significantly.


     10:05 pm on Dec 7, 2002 (gmt 0)

    While i can give the nod on the css/fewer results per page/shrink images (or higher jpg compression is possible), mod_gzip will make a huge difference for text based sites. The load is indeed quite small, except for one caveat: when you start to reach your maximum threshold, it is spooky painful.

    On on of the search engine services i run, i found this problem the hard way. Basically around the peak times of day, the load average on the machine would go from it's normal "1 or 2" to 100 or 200 (obvoiusly causing all sorts of problems). After some investigation, it became clear that mod_gzip wasn't fast enough to spew out the pages at the rate I needed them: this was a runaway cpu resource problem. Basically while it is workin on compressing some pages, it would get requests to compress more pages before it was finished with the first ones, resulting in the server to collapse. The temporary fix was to change the sizes of files which i allowed it to compress (such that it would effectively be comrpessing fewer files with a lower level of compression - i think it defaults to a level of 6).

    This first reared it's head around 35 [compressed] php pages per second from a 1xP4-2.2Ghz machine. Again, the big issue there is the cliff effect: you can't really tell how close you are, but when you step over it, it's painfully obvious! I since upgraded the machine to a 2xP4-2.4Ghz and have sustained rates of over 50 pages/second without any issues.

    I would think that most forum's wouldnt need this type of throughput, but I would strongly encourage anyone looking to deploy mod_gzip to at least profile their server (i.e. stress test) to get a feel for where the breakdown is with your configuration.


     3:16 am on Dec 8, 2002 (gmt 0)

    >More than 50% of your daily uniques from search engines?

    If that's true, it means that more that 50% of your visitors are first timers and you want the fastest loading page possible. That first page view for a first time visitor is the most critical. Speed isn't the only thing, it's everything (or vice versa!?).

    It's that all important out of the box experience. On the web, that means speed is critical. Slow loading pages and they will resist going farther into the site. Fast loading pages, and they are more likely to stick around and view more.


     3:58 pm on Dec 8, 2002 (gmt 0)

    turk182 - how do you explicitly make js css and images cacheable?


     4:18 pm on Dec 8, 2002 (gmt 0)

    how do you explicitly make js css and images cacheable?

    If you use an apache server, have the following added to you configuration. It will add the necessary headers to cache images and javascript files.

    # Expire in one week
    ExpiresActive On
    ExpiresByType image/gif A604800
    ExpiresByType image/jpeg A604800
    ExpiresByType image/png A604800
    ExpiresByType image/x-icon A604800
    ExpiresByType application/x-javascript A604800


     6:08 pm on Dec 8, 2002 (gmt 0)

    And on Windows server? Thanks!


     10:21 pm on Dec 8, 2002 (gmt 0)

    [quote]how do you explicitly make js css and images cacheable?[quote]

    In IIS is as simple to put them in another directory and from the IIS panel select the properties of that directory (right button over it). There's a section where you can modify all the properties related to these files, like how long should be cached before fetching them another time (I don't remember exactly what section was, and now I'm at home and I don't have a IIS installed ;-)


     1:29 am on Dec 9, 2002 (gmt 0)

    Thanks for the suggestions... plenty of things to add to my "to do" list. I did some analysis on the site that prompted the original post, and I found that a few "bad actor" pages were major culprits: very high views and very big pages. I've done some major surgery, and by tomorrow I'll see the first results of my handiwork.

    In the long run, though, I think some of the suggestions about compression and caching will have a big impact across the entire site.


     7:22 am on Dec 9, 2002 (gmt 0)

    kmarcus, I'm curious if you took at look at what was causing the load increases. Was your processor hitting 100% or were you running out of RAM (thrashing)? I kinda find it hard to believe that the gzip would cause the load problems. It may have been something else.

    As for the bandwidth stuff:
    1) Cut any animated gifs completely. Many forums have some emoticons that are animated. Its a huge waste of bandwidth.
    2) Don't just reduce the number of messages per page. Experiment with it. Remember, users may end up viewing more pages if you reduce the messages per page, and this has the massive overhead of the rest of your HTML template.
    3) Force users to register before they can view the forums (not the most friendly thing in the world, but it will keep down page views).
    4) There are numerous HTML tricks you can use to reduce bloat. Definitely use css for redundant styles, but you can also do things such as reduce colors to 3 characters instead of 6 (#EE9966 -> #E96). You can detect a browser on the server side, and if it supports advanced CSS, you can do away with all the tables on the page and run the layout w/ external css (this of course has an even better effect on multiple page views).
    5) Kill comments/alt tags.
    6) Keep a page view count on every ip and restrict them to say 50/day or something crazy high to cut out bad guys.


     9:06 pm on Dec 9, 2002 (gmt 0)

    yes, the cpu hits ~100%. i'd encourage you to give it a whirl. The stats are above but around 40 or so pages/second is where my hardware started having the big death spiral problems.


     10:43 pm on Dec 9, 2002 (gmt 0)

    I got pretty excited when I tried out the mod_gzip tester that was recommended above - big size reductions! Unfortunately, my host advised me that they don't support mod_gzip. (Cuts into their bandwidth surcharge too much? ;))


     10:21 am on Dec 10, 2002 (gmt 0)

    A little "off-topic" article from NYTimes [nytimes.com] about how price is limiting demand for broadband.

    This 37 message thread spans 2 pages: 37 ( [1] 2 > >
    Global Options:
     top home search open messages active posts  

    Home / Forums Index / Code, Content, and Presentation / HTML
    rss feed

    All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
    Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
    WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
    © Webmaster World 1996-2014 all rights reserved