Larger the file, slower the Apache sending it

[teasers]

I have a car-related site and have dozens of .wmv videos on the website.

During the last 2 years the files began getting larger in size, i.e. between 100-200 Megabytes, instead of 10-20 Megabytes compared to 2-3 years ago.

I noticed that Apache 2.2 began serving those files slower. The bigger the file, the slower it serves.

200 MB file: speed 30KB/s
100 MB file: speed 50KB/s
30 MB file: speed 150KB/s
1 MB file: speed 450+ KB/s

I am on a dedicated server which belongs to myself, so I can see the server stats, there are no problems on the server end.

- I am on a 100 MBit port, no restrictions, no topping on bandwidth.
- Looging into the server via SSH and then "top" shows a maximum of 0.05 load. No load on server.
- I even tried an OCZ Solid State Hard Drive. No changes.
- I have an 8Mbit internet connection at home. When I FTP and try to download the same video, the speed is the same around 50KB/s and never go up.
- If I download using 8-10 threads (my FTP software has a multi-thread downloading feature, which can download up to 20 files at the same time), then my download capacity is maxed, like 700-800KB/s or similar.

I have CentOS on the server. Is the problem caused by Apache, or CentOS? Maybe it is a precaution for restricting a single user not topping up the server's bandwidth. There must be some configuration for this, but where? Any help will be appreciated.

[Frank_Rizzo]

200 MB file: speed 30KB/s
100 MB file: speed 50KB/s
30 MB file: speed 150KB/s
1 MB file: speed 450+ KB/s

Is the the download speed to your PC or are all users getting those speeds.

It could be the case your ISP is limiting the speed of bigger files.

Caching is a possibility. Small files are cached, 200Mb files are probably not and thus are being read from the disk (which presumably is pulling all over the place). Try running a download at the quietest time.

Was the SSD a value range? Some are slower than traditional hard disks.

[teasers]

-Was the SSD a value range? Some are slower than traditional hard disks.

Really? I didn't know that.. It was OCZ Vertex 120GB, but I don't know if it was top-of-the-line or not.


-It could be the case your ISP is limiting the speed of bigger files.

Nice question. I will send a message to users and will request them make a download test. The ISP limitation is a possibilty, you're right. I will test that. Thank you for writing.

[jdMorgan]

This is commonly called "traffic shaping" or "bandwidth shaping" and is an attempt by the ISP to make sure that no-one can "hog" all the bandwidth.

So, if this is your ISP's doing, then small files can be downloaded "at up to 8Mbps" (which likely appeared in the marketing material that you received prior to signing up), but once so many bits and bytes have been transferred within a given period of time, then the 'shaping' kicks in and your bandwidth is throttled down.

You might also try downloading some large files from other sites -- youtube, dailymotion, etc., and watching the rate of change of the "download progress bar" over time. If it goes from zero to 25% in seconds, but then takes longer to get to 50% and even longer to get to 75% and 100%, and this occurs on several sites, then it would indicate that your ISP is throttling your connection.

Jim

[teasers]

Well, got responses from 4 people:

U.S. 50 KB/s
Poland 25 KB/s
Turkey 80 KB/s
Poland 800 KB/s

I am not sure if the last one the guy said is true or not. He says he is on a 7 mbit line, so that means he really maxed out his line. But why so many people are having a download speed that low, between 25-80 KB/s? All ISP's shaping bandwidth, but only those 800 KB/s guy's is not doing it? Anyway, everything will be more clear as I get more results.

In my case, the download speed begins with 80 KB/s and continue as 80 KB/s.. Nothing like very fast start then decrease in speed. It is just 80 KB/s both at the beginning, in the middle and at the end.

The server is in the UK, and when I test my download speed using some competitors' test files (100mb.bin etc), they are almost the same. So, is it something like UK's bandwidth limit is topped? No..

Is it possible that the carrier like "Linx" or something topping out, or shaping the traffic? Because, while is so rare, I sometimes see 700KB/s speeds i.e. downloading a trial software, or on Adobe updates, etc. Anyway, more tests are needed. I also have to make some download tests from different locations, to check if I get 800KB/s from a single file.