Forum Moderators: phranque

Message Too Old, No Replies

Too much traffic?

         

Scooter24

12:13 pm on Oct 19, 2002 (gmt 0)

10+ Year Member Top Contributors Of The Month



I have a huge web photo gallery site (currently 380 MB, 2100 photos) and am on a package for 500MB webspace and 20 GB monthly traffic. A few days ago I finished adding 270 new images to my Thailand photo gallery. I posted a message in the rec.travel.asia and soc.culture.thai newsgroup and - gulp - traffic shot up from an average of 500 MB/day to 1.1GB/day. Traffic has been at 1GB/day for the past three days. Should this go on like this I would exceed my monthly quota of 20GB (and then it gets really expensive - $10 for each additional GB).

What I'd like to know what a "normal" traffic for a 400 MB site is. Do I need more than 20GB? I might get an additional 10 or 20GB package if necessary.

davidku

8:20 am on Oct 21, 2002 (gmt 0)

10+ Year Member



Talking about .htaccess. Try this way to INCREASE your traffic

RewriteEngine On
RewriteCond %{HTTP_REFERER} !^http://www.mydomain.com
RewriteCond %{HTTP_REFERER} !^http://mydomain.com
RewriteCond %{HTTP_REFERER} !^http://123.456.789.0

RewriteRule /* [mydomain.com...] [R,L]

Substitute someimage.jpg with your site logo probably showing the URL. This will turn the traffic from image hijackers into your own. Neat isnt it?

Gorufu

10:03 am on Oct 21, 2002 (gmt 0)

10+ Year Member



RewriteEngine On
RewriteCond %{HTTP_REFERER} !^http://www.mydomain.com
RewriteCond %{HTTP_REFERER} !^http://mydomain.com
RewriteCond %{HTTP_REFERER} !^http://123.456.789.0
RewriteRule /* [mydomain.com...] [R,L]

This will turn the traffic from image hijackers into your own. Neat isnt it?

davidku

It will create an endless loop and no-one will be able to access the site. When someone tries to enter or access the site they don't have [mydomain...] as the refering URL so they are blocked. They are also blocked from someimage.jpg because they are unable to send the correct refering URL

Check your sticky mail.

How about international and other portals/proxy cache servers who do not use Mozilla/3.01 (compatible;) as their UA? While I understand this code accomplishes more efficient screening for UA spoofing, I still am hesitant to use an accross the board block with unknown variables when I get significant international referrals.

keyplyr

RewriteRule .*\.(gif¦jpg¦png)$ - [F,NC]

Images only are blocked from hotlinking/downloading
The site can still be entered in the normal manner, via a search engine link, bookmark, etc.

Over 65% of my visitors are from Asian countries and these countries also use Mozilla/3.01 (compatible;) for the proxy servers.

I spent considerable time studying my access.log files for all access without a refering URL. In nearly all cases the site was rapidly downloaded and the visitor would not have had enough time to read any webpage on the server.

musicales

10:25 am on Oct 21, 2002 (gmt 0)

10+ Year Member



I'm interested in this blank referrer but not sure I understand it. My site gets a good 10% showing up as no referrer in my stats. Are you saying they may not be real visitors or may be downloading using software other than a browser? I assumed it just meant they'd come straight to my site, eg as a home page.
Thanks

Gorufu

11:15 am on Oct 21, 2002 (gmt 0)

10+ Year Member



I assumed it just meant they'd come straight to my site, eg as a home page.

Hi musicales

When a visitor enters your site directly they do not have a refering URL. If they enter from a search engine or link from another site the other site's link will be the refering URL.

Refer to example below for a direct visitor. There is no refering URL for the page the visitor entered. All the images accessed from index.html have the correct refering URL "http://domain.com" so they are returned to the visitor.

GET /index.html HTTP/1.1" 200 931 "-"
"Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)"
GET /one.png HTTP/1.1" 200 842 "http://domain.com/index.html"
"Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)"
GET /two.jpg HTTP/1.1" 200 18515 "http://domain.com/index.html"
"Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)"
GET /three.png HTTP/1.1" 200 1676 "http://domain.com/index.html"
"Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)"

In the following example there was no refering URL for the images and a 403 error was returned to the visitor. mod_rewrite has blocked access to the images because there was no refering URL from index.html

GET /index.html HTTP/1.0" 200 4244 "-"
"Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"
GET /one.png HTTP/1.0" 403 0 "-"
"Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"
GET /two.jgp HTTP/1.0" 403 0 "-"
"Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"
GET /three.png HTTP/1.0" 403 0 "-"
"Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

musicales

11:39 am on Oct 21, 2002 (gmt 0)

10+ Year Member



Gorufu thanks. So your checking only to make sure images have a refering page, but not the pages themselves right?

Gorufu

12:11 pm on Oct 21, 2002 (gmt 0)

10+ Year Member



So your checking only to make sure images have a refering page, but not the pages themselves right?

musicales,

Yep, thats correct. You can also stop hotlinking to pdf, mp3, zip, etc, by addding the extensions to the Rewrite Rule.

RewriteRule .*\.(gif¦jpg¦png¦pdf¦zip¦mp3)$ - [F,NC]

quotations

5:03 pm on Oct 21, 2002 (gmt 0)

10+ Year Member



With regard to your "how much is enough bandwith" question, the answer (as always) is:

It depends.

I have a 43 MB site which is mostly text and which has been live for less than a month and it is getting close to exceeding the monthly allocation of 20GB.

It isn't even showing up in Google yet.

I will have to split it soon.

I would start splitting your image files up between multiple sites if I were you. Put the 100 most popular images on one site and the less poplular images on other sites. Then as the "most popular 100" exceeds its bandwidth you will probably only have to split that site and not have to worry too much about the others.

Once you have the most popular under control, you can concentrate on making the other images more popular.

Bill Austin
(yes, that one)

Scooter24

9:19 pm on Oct 21, 2002 (gmt 0)

10+ Year Member Top Contributors Of The Month



I have a 43 MB site which is mostly text and which has been live for less than a month and it is getting close to exceeding the monthly allocation of 20GB.

It isn't even showing up in Google yet.

Where does all the traffic come from?

I would start splitting your image files up between multiple sites if I were you.

I called my provider today. They said, if I get another package I'm not allowed to use the additional bandwidth under my domain. So I'll have to start splitting the site soon. I'll probably start putting the JPEG files of the full size images (which make up the bulk of the webspace) into the new domain, so that the URLs of my site won't change and nobody (visitors, search engines) will notice the modification.

sssami

10:09 pm on Oct 21, 2002 (gmt 0)

10+ Year Member



In Response to, Too much traffic, how would you go about advertising you site and getting so much traffic to it as i just have no idea at the moment

sssami ;)

IanTurner

11:49 pm on Oct 21, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Get a second account under a different domain and then point a sub domain of the initial domain to the second domain.

e.g.
www.domain1.com - main domain account1

www.domain2.com - second domain account2

Pointer images.domain1.com -> www.domain2.com

davidku

12:07 am on Oct 22, 2002 (gmt 0)

10+ Year Member



Hi Gorufu,

Thanks for correcting.

May I know where I can get some resources to learn up the .htaccess programming?

daesasitorn

1:34 am on Oct 22, 2002 (gmt 0)

10+ Year Member



Scooter24: What I'd like to know what a "normal" traffic for a 400 MB site is

Interesting, I am doing something quite similar, running a photo library on a website. I've got about 11,000 images in there, and I've almost used up my 600 Mb space. I have an average of 1,200 visitors a day and each of them visits about 7.6 pages. My traffic is about 10 Gb (1,000,000 hits) a month. I always keep my images size between 30 - 70Kb, but like you image quality is so important, at 415 pixels, I only use 70% jpg compression, so the quality of images is still there). I think your images are much too big and it takes a long time to load, you might find that having images smaller, people will stay longer and see more of your site. I've studied site statistic very seriously, and that is one of many other things I have learnt.

By the way, could you send me a sticky mail for url of your site. I am very interested to see your Thailand gallery. I kind of home sick, have not been back home for quite some years, it would be nice to see the pictures.

gilli

9:20 am on Oct 22, 2002 (gmt 0)

10+ Year Member



I can't believe the figures being quoted here. Here in Australia a *good* rate for bandwidth is $0.08 per MB with some companies throwing in free bandwidth to a point and then hitting you with $0.20 - $0.35 per MB.

Did someone say something about the @rse end of the world?

Scooter24

12:59 pm on Oct 22, 2002 (gmt 0)

10+ Year Member Top Contributors Of The Month



Here in Australia a *good* rate for bandwidth is $0.08 per MB with some companies throwing in free bandwidth to a point and then hitting you with $0.20 - $0.35 per MB.
Did someone say something about the @rse end of the world?

$0.08 per MB - that would be $80 per GB. $0.20 - $0.35 per MB is $200-350 per GB. You call that a good rate? Maybe Australia is a good place to set up an ISP business.

I just bought an additional package - 100MB webspace and 7GB of traffic for $4/month. Will put the Thailand JPEGs there.

Scooter24

1:14 pm on Oct 22, 2002 (gmt 0)

10+ Year Member Top Contributors Of The Month



Too much traffic, how would you go about advertising you site and getting so much traffic to it as i just have no idea at the moment

My site ranks pretty well with Google and most country galleries are listed in DMOZ. For example the Malaysia gallery ranks Nr.2 in Google for 'Malaysia pictures' (used to be Nr. 1 until last month). Every day the Googlebot visits between 1000 to 2000 pages to check if I made changes. Sometimes new pages of my site appear in Google within a few days.

I also suspect that I get quite a number of repeat visitors. The site is huge and you can't visit it all in one session.

shady

4:57 pm on Oct 22, 2002 (gmt 0)

10+ Year Member



I've found a deal which may be of interest to you, offering the following:

35 Gig of disk space
50 Gig traffic / 10 million hits
$15.95 per month

They also have 24/7 tech support, with freephone 1-800 telephone support available 7AM PST to 9PM PST, Monday - Friday

I think its only a single domain, but it looks like a good solution for your kind of site.

I'll sticky the details to you. If anyone else has a need for this kind of host, please let me know.
btw I'm not affiliate with them, just looks like a good deal!

daesasitorn

5:47 pm on Oct 22, 2002 (gmt 0)

10+ Year Member



Shady : could you sticky details of this host to me? Have you got any experiences with them and how realiable are they?

shady

5:58 pm on Oct 22, 2002 (gmt 0)

10+ Year Member



I do not have any experience of them. I have a number of "reseller" accounts at other companies, which are more suitable for me.
I think these people offer a good deal for a single domain which has high b/w or requires a large amount of webspace.

The deal was found by a friend of mine, who I believe will be opening an account. I have no information, regarding the service quality. I am not recommending them - it just looks like a good deal!

gilli

12:03 am on Oct 23, 2002 (gmt 0)

10+ Year Member



Scooter24:

I was talking in Aussie dollars so you can cut those figures in half, but I was being very sarcastic when I said they were *good* rates.

I don't think Australia is a very good place to start up an ISP business unless you went in very big. The thing is, almost all the ISPs are paying very high prices for bandwidth themselves, except for the one or two largest - who actually have the physical infrastructure.

keyplyr

5:55 pm on Oct 28, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month




You guys pay too much.

  • Disk Space (Raid Storage) 250 MB
  • Data Transfer/Bandwidth 25 GB/mo.
  • POP3/IMAP Email & SMTP 250
  • Catch-All Email/Email Forwarding
  • 24 x 7 FTP Access
  • FTP Users 10
  • CGI-BIN, SSI, .htaccess, Cronjobs
  • Web-Based Email
  • PHP4, Perl5, Sendmail
  • FrontPage 98/2000/2002 Ext.
  • Detailed Webstats Report
  • Access to Raw Log Files
  • SSL (Secure Server) FREE!
  • 24/7 Email Support
  • Telephone Support
  • DNS Control ( A record, Cnames, MX records)
  • Stream Server FREE!
  • MySQL Database

    $7.77 a month
  • dingman

    2:10 am on Oct 29, 2002 (gmt 0)

    WebmasterWorld Senior Member 10+ Year Member



    <edit>I should have sticky-mailed. Sorry. I will now resume thinking before I post.</edit>

    Scooter24

    8:49 am on Oct 29, 2002 (gmt 0)

    10+ Year Member Top Contributors Of The Month



    $7.77 a month

    Sticky-mail me the URL

    This 52 message thread spans 2 pages: 52