Welcome to WebmasterWorld Guest from 54.144.7.239

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

Finding bandwidth hogs

     
9:30 pm on Apr 24, 2012 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 22, 2003
posts: 100
votes: 0


Our site routinely goes over our host's allotted monthly bandwidth, not that big of a deal but we pay extra for the additional bandwidth.

Is there a tool (either desktop or server-based) that can analyze a month's worth of log files and come up with a breakdown of the top files by bandwidth, along with referrer info so I can verify that the traffic is legit? Also useful would be user agent info so I can determine if there are useless bots wasting bandwidth too

Our host has a plesk stat log analysis tool which is handy, but it doesn't get specific enough (it tells me that jpg images take x amount of bandwidth, but not which images in particular or user agent / referrer info for those images)

Thanks in advance for any suggestions
10:57 pm on Apr 24, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 24, 2002
posts:894
votes: 0


I would suggest that you start by comparing your raw log files with everything you can find in [webmasterworld.com...]

You will be amazed at how much bandwidth is wasted
11:02 am on Apr 25, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 27, 2002
posts:765
votes: 0


There are a few 3rd party products around that even in their free version can give you a good indication of what is using up your bandwidth. Usually they used for optimizing load time, but I found that applying the recommendations also resulted in reducing bandwidth significantly. (I will send you a link to a tool via sticky - not affiliated but happy user of that tool).
11:07 am on Apr 25, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 27, 2002
posts:765
votes: 0


If possible, try to switch on compression for HTML / CSS / JS files with GZip. Saves a lot!
4:26 pm on Apr 25, 2012 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 10, 2004
posts: 390
votes: 12


In addition to compression, image cache headers are your friend!
11:54 am on Apr 29, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Apr 30, 2007
posts:1394
votes: 0


Maybe something like piwik web analytics will help.
7:37 pm on Apr 30, 2012 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 10, 2004
posts: 390
votes: 12


Do you have shell access and raw Apache server logs? If so, I can give you some onliner awk commands which will do exactly what you need.
9:32 pm on Aug 29, 2012 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 22, 2003
posts: 100
votes: 0


I just noticed this thread in my control panel and that I never responded to anyone, I apologize, my email notifications go to an address that I almost never look at.

anyway I did do some log analysis and found a couple of things that were legit, so it does seem like it's ok

motorhaven if you still have those awk commands though I'd definitely give it a shot, we have about 2 months of logs on a rolling basis

thanks!