Exceeded allocated monthly traffic(attacked)

preventing future attack

[hamids54]

hi

we recieved email about Exceeded allocated monthly traffic.
it says Please take note that the allocated traffic included with your Budget-Performance (calculated by GB of traffic) has been exceeded Traffic generated for the month of November to this day: 1698.331GB Allocated monthly traffic: 1500GB

we always use less than 100 gigs per months .I think our server has been attacked .my question is what we must do to prevent it for future?

[tangor]

May not be an attack... are you doing any quality control over the number of robots that hit your site? Reduced my bandwidth significantly when I went whitelist only!

[hamids54]

as i said we always use less than 100 gigs...do you think
robots use 1500 gigs bandwidth? it is not explainded

although I am not sure attack is the cause.

[jdMorgan]

The first thing to do is to looks at your 'stats' -- The server access statistics provided by your host. As tangor points out, your site may be getting hits by many, many search engine robots -- and also by many malicious automated user-agents as well.

On the other hand, maybe your images and/or video files are being hotlinked -- showing up on other sites' Web pages, but being loaded from your server.

Or perhaps you've just experienced an up-tick in traffic, but your site has too many images, JavaScripts, and flash files embedded in it.

Or maybe you're not sending cache-control headers properly.

... we can only guess, as Web sites are not simple things...

The first thing is to identify the specific cause(s) of your bandwidth overage, and then you can investigate appropriate solutions.

Jim

[hamids54]

thank you

I use sitemeter and google analytics and awstats.all are normal as before.when I enter control panel of server it shows 45 gb for monthy traffic.is there any especific stats
on server other than i mentioned? my server is linux.

[jdMorgan]

> The server access statistics provided by your host...

Jim

[hamids54]

I did chek up server access statistics provided by host

it was true.we exceeded monthly traffic

I did chat with datacenter...they said if it happens another time
they will investigate.at now i have to pay 50 dollars for 170 gigs
that has been exeeded vainly.I worry about another time it reaches 10000 gigs and have to pay very very much money vainly again.we have only 2 sites there without any video or photos.just
some articles that has been pdf

[wilderness]

Rather than focusing upon server/host provided stats, which are configured under the umbrella of serving multiple customers (as opposed to specific customers and specific websites)?

perhaps your initial exploration should begin with your websites "raw logs" (raw visitor logs).
View these logs will allow you to draw your own conclusions as to what IP range and or User Agent was being utilized for this increased traffic.

[hamids54]

thanks all

I got server just for my own site

I looked at network usage.it exceeded monthly traffic(1500 gigs)
yesterday was 1600 gb today is 1700

I dont have any videos or photos on my site just some articles that have been pdf from 5 years ago.I am confused what is the cause?

I myself don`t know about server managing...the one who have done it before does`t liks to works for me it seems...

[My hosting company] replied me if i suspect DDoS attacks it is better for me to use their [security, firewall, and DDOS-protection] services

I replied them I am not sure attacking is the cause...I want to investigating about it...

[edited by: jdMorgan at 4:52 pm (utc) on Mar 3, 2010]
[edit reason] No specifics, please. [/edit]

[tangor]

Reminder once again... have you looked at the RAW LOGS for your server? You'll see entries that look something like this:

65.55.105.xx - - [01/Jan/2009:00:04:26 -0700] GET /robots.txt 200 1293 - msnbot/1.1 (+http://search.msn.com/msnbot.htm)

Which reads "IP - - Date Time Request Status Size Referer UserAgent"

There are other log formats which give the same info, but in general you get a wealth of information for making decisions about who to let in, who to keep out, or who to ban (if necessary).

[hamids54]

thank you

my server manager did cheked up log files and he founds nothing.he posted [an] email [to our hosting company] [pointing out that the backup software apparently has a bug which is causing the huge bandwidth usage].

[edited by: jdMorgan at 4:50 pm (utc) on Mar 3, 2010]
[edit reason] No e-mail quotes, please. See Terms of Service. [/edit]

[jdMorgan]

The reason I suggested reviewing the 'stats' above is because it presents different 'views' of the server activity. For example, top-bandwidth pages, requests sorted by user-agent, etc. Looking at these pretty graphs, it might be easier to spot what the 'major' problem is.

After finding the major problem, then you can dig into the raw server access logs looking for those problematic requests.

It seem like your server manager thinks the problem is the "r1soft backup" program. So until you can prove otherwise, I'd advise turning off the backup feature, or asking your host to turn it off.

Jim

[hamids54]

thank you Jim

I'm awaiting a reply [from my host].

from 12.00 wed I have had normal situation.when i look at Bandwidth usage details (demographic, I mean) from last week I have had 3 times exceeded BW. then waiting if it happens again I will take [the backup] program off or ask [my host] to fix the problem.

[edited by: jdMorgan at 4:55 pm (utc) on Mar 3, 2010]
[edit reason] No specifics, please. [/edit]

[jdMorgan]

You might also want to ask them if it's possible to configure that backup program more specifically -- For example, to configure it to only back up once a week. Or configure it to NOT back up gif and jpeg images and other files that almost never change. Or to not backup the 'stats' pages or the raw server log files, etc.

In other words, it may be possible to tell the backup software to back up less frequently and/or to only back up the stuff that you actually need to keep a backup of, and then you can manually make and keep backups of the static, rarely-changing files only on your own PC.

Jim

[hamids54]

I myself ask them to have daily back up automatically.I pay for it 50 dollars per month.

after fixing this problem I will ask them to configure it more
specifically.i don`t know about it.

do you think this software gets huge BW?

[jdMorgan]

If your site is dynamic and correct cache-control headers are not set, it could very well be backing up every single URL every day. In that case, yes, the backup would consume huge bandwidth.

Be more selective. Do you really need everything backed up every day? Probably not.

Jim

[well]

You can try [a] firewall for protection against DDoS attacks.

[edited by: jdMorgan at 4:55 pm (utc) on Mar 3, 2010]
[edit reason] No promotions, please. See Terms of Service. [/edit]

[hamids54]

thank you friends

at now my server is stable and normal situation

my provider accepted [that the backup software] was its cause. they fixed it. it seems there was a bug.
at future month I will want them to take [the backup software] off. I don`t need it and can`t pay
its cost too (at present I don`t earn money from my site). I will ask someone to download my data weekly on his hard by high speed connection. it is about 1 gbs. [The suggested] firewall is too expensive for me.

[edited by: jdMorgan at 4:57 pm (utc) on Mar 3, 2010]
[edit reason] No specifics, please. [/edit]

[tangor]

hamids54... thanks for update (and sorry for late reply... I was in hospital). Some hosts add in backup traffic as bandwidth, some do not. I do all my own backups, actually have three versions of site(s), a desktop, laptop, and the web host so do not pay for onsite backups via host. For all others it makes good sense to find out if host backup is inclusive of site bandwidth as that can lead to unexpected costs.

[hamids54]

thanks tangor

at now I don`t have problem with BW issue.I asked my provider to remove the software.I want to download one copy of my data to my desktop but my connection speed doesn`t lets me.my server gets copy of my site for three
last days.I want to use offsite backup websites that needs
low costs... about 5 $ per months