consumed too much bandwidth

sudden spike on bandwidth

[squallions]

My small website is based on phpbb forum. It consumed about 300 MB a day before but since OCT 12, it suddenly increased to about ~1300MB/day and for the last three days, it used about 2GB.

Yes, the number of visitors also increased as well from 300 a day to ~900 a day. No big file serves on my website. Mainly php, gif and swf (small game). I look at the log and most requests are php and gif.

Is this normal for a site with 900 visitors/day? Is there any other factors that caused bandwidth to increase like that? My friend said someone might play sql injection on my site and that cause bandwidth to increase. Is that true?

Thank
sql.

[gertrijs]

If I were you, I would check if there is not some movie (or other big file) hidden somewhere on your host.

Gert

[squallions]

I have check and there is no movie or other big file.

[Jon_King]

I think your traffic at 900 uniques a day is totally possible and really not that far out. If you've got decent rankings or return traffic I would expect what you are receiving. Check your log files and see what pages are high traffic and that will answer much about your bandwidth issue. IMHO

[dataguy]

Are you sure that part of the spike isn't due to excessive crawling by the Gbot, MSNbot, etc. that's being seen on many sites? This can spike bandwidth, though crawlers don't request image files....

[McElvoy]

For a site with that little traffic, it should be pretty easy to open your logs with Excel, sort by file bype, subtotal the bandwidth column, and get an idea of what kinds of files (or individual files) have caused the difference.

Tripling the traffic (300 to 900) but 7 times the bandwidth seems a little off.

But unique visitors isn't a good stat for this kind of comparison. Did visits also increase by x3? or by x7?

[Rosalind]

Tripling the traffic (300 to 900) but 7 times the bandwidth seems a little off.

That sounds about right for a forum, because with three times as many members they will all be finding more people to talk to, and therefore looking through more threads. If each unique visitor is consuming 1-2MB worth of bandwidth, it's worth examining how many pages it takes them to clock this up. You could probably make quite a saving by reducing the file size of a handful of images.

[SkyDog]

Either bots, or most likely some pr*ck is hotlinking one or more of your files. Check you web logs.

[squallions]

I have check the log file and it's about 22MB so I find it hard to find which page is most frequently visit. How do you use excel to view these kindda log?

66.157.60.125 - - [02/Nov/2004:00:02:33 -0800] "GET /vforum4.html HTTP/1.1" 200 12399 "http://www.mysite.net/forum.php" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; .NET CLR 1.1.4322)"
66.157.60.125 - - [02/Nov/2004:00:02:34 -0800] "GET /images/icon/icon18.gif HTTP/1.1" 200 1085 "http://www.mysite.net/vforum4.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; .NET CLR 1.1.4322)"

Most request on the log as I glanced through them are .html and .gif.

According to the forum visit: ~900 visits a day
According to Awstats so far for November:

Unique visitors: 4376
Number of visit: 7041
Pages: 159779 (22.69 pages/visit)
Hits: 894205 (126.99 hits/visit)
Bandwidth: 14.22 GB (2117.8 KB/visit)

For the bot thing,
Google: 11679 - 258.84 MB
MSN: 3197 - 153 MB
Inktomi Slurp: 835 - 7.21 MB
....and other small bots

Is this normal? Yesterday was about 1.8GB BW consummed. At this rate, my site will die on Nov 20. :( Have only 40GB/month. :(

[encyclo]

Glancing at the figures, they look quite reasonable, including the bandwidth use. Forums are bandwidth-intensive at the best of times, and when you get more visitors, you get more interactions, more threads, more private messages, and visitors stick around longer because there is more to read.

Your host should be running a stats analysis program accessible from the control panel: something like Webalizer or Awstats: those tools will do the parsing of the log files and give good summaries as to which files are the most popular. If it is your graphics, get them optimized as much as possible (and unless you're using transparency, jpg is almost always smaller than .gif).

You options are to pay for more bandwidth from your current host, or move to a host which offers a better bandwidth allocation and doesn't cut off your site when you hit a limit.

<added>Rereading, I see you've got Awstats: that will tell you which files are the using the most bandwidth.</added>

[squallions]

According to Awstat:

gif: 653110 - 532.83 MB
html: 96812 - 4.62 GB (html exist because I convert php to html using the htaccess thing)
php: 45015 - 8.75 GB
jpg: 40321 - 120.29 MB
..and other small extension: js, etc...

Buying more bw is expensive :) I'm think getting another host with more bandwidth.

Thank all.

[instinct]

If phpbb and your host support gzipped html (google it) then I would see about how you can enable it.

Should save quite a bit on bandwidth.