homepage Welcome to WebmasterWorld Guest from 184.73.40.21
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / WebmasterWorld / Webmaster General
Forum Library, Charter, Moderators: phranque

Webmaster General Forum

This 52 message thread spans 2 pages: 52 ( [1] 2 > >     
Too much traffic?
Scooter24

10+ Year Member



 
Msg#: 1427 posted 12:13 pm on Oct 19, 2002 (gmt 0)

I have a huge web photo gallery site (currently 380 MB, 2100 photos) and am on a package for 500MB webspace and 20 GB monthly traffic. A few days ago I finished adding 270 new images to my Thailand photo gallery. I posted a message in the rec.travel.asia and soc.culture.thai newsgroup and - gulp - traffic shot up from an average of 500 MB/day to 1.1GB/day. Traffic has been at 1GB/day for the past three days. Should this go on like this I would exceed my monthly quota of 20GB (and then it gets really expensive - $10 for each additional GB).

What I'd like to know what a "normal" traffic for a 400 MB site is. Do I need more than 20GB? I might get an additional 10 or 20GB package if necessary.

 

fathom

WebmasterWorld Senior Member fathom us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 1427 posted 12:24 pm on Oct 19, 2002 (gmt 0)

Well your problem "May be a gold mine".

If your Thailand photo gallery doubles your bandwidth consumption, your traffic must be up as well and this seems like a really good "new" niche. (there are also pitfalls like bandwidth thieves, so do some investigation).

Look for opportunties here and "Yes" I would definitely consider a more robust package for bandwidth (or reduce the need for bandwidth, smaller file sizes, thumb nails, off site re-directs, etc.)

NeedScripts

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 1427 posted 8:48 pm on Oct 19, 2002 (gmt 0)

(and then it gets really expensive - $10 for each additional GB).

That is really expensive. You can easily get deals from good companies for about $ 1 to $ 3 per GB Bandwidth.

JonB

10+ Year Member



 
Msg#: 1427 posted 8:54 pm on Oct 19, 2002 (gmt 0)

what about some image compressors? with so many picutre few kb will add one every pic.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 1427 posted 9:30 pm on Oct 19, 2002 (gmt 0)

I have a large photo gallery as well. I cut my bandwidth by 20% after I:
  • started optimizing the image files
  • posted a notice to please not link to photos
  • put an image wild card switch in .htacces to replace the remotely linked images with an embarrassing announcement (he he)
  • banned directory browsing
  • banned known file harvesting UAs and bots

    If I was to build another gallery from scratch, I think I would have my images served on the fly, with only thumbnails in the HTML.

  • Shakil



     
    Msg#: 1427 posted 9:59 pm on Oct 19, 2002 (gmt 0)

    For that sort of bandwidth you should be looking to pay $1 > $2 per GB of traffic.

    And I agree with that sort of traffic, there HAS to be a way to make some $$$s

    Shak

    Scooter24

    10+ Year Member



     
    Msg#: 1427 posted 10:59 pm on Oct 19, 2002 (gmt 0)

    Thanks for all replies. To address the points you made:

    Currently I'm on a package 500MB/20GB for $15/ month. Exceeding the traffic quota costs $10/GB, but you can also get another 500MB/20GB package for $15, so it's not so terrible.
    I have disabled hot linking (with .htaccess), disabled directory browsing, banned most known download agents. Even inserted code which automatically adds a deny from xx.xx.xx.xx line to .htaccess when somebody tries to download the site.
    I'm trying to understand what an appropriate traffic for a 500 MB site is. I suspect that a well established 500 MB site generates considerably more than 20GB/month of traffic, maybe something between 30 and 50GB, but I can't tell as I'm relatively new to the web.

    The big problem is that at the moment I'm not translating this traffic into income. There is no advertising for instance. I just became an Amazon associate and will soon offer some books (travel guides) for sale, but I don't know how much revenue this will generate. I've been told that banner ads have extremely low click through rates and therefore generate very little money.

    So, I don't know if I should be happy if traffic increases. I've even stopped promoting the site.

    shady

    10+ Year Member



     
    Msg#: 1427 posted 11:10 pm on Oct 19, 2002 (gmt 0)

    This is indeed a large amount of bandwidth!
    What is your average graphic size? How many hits/day are you receiving?

    fathom

    WebmasterWorld Senior Member fathom us a WebmasterWorld Top Contributor of All Time 10+ Year Member



     
    Msg#: 1427 posted 11:42 pm on Oct 19, 2002 (gmt 0)

    The big problem is that at the moment I'm not translating this traffic into income. There is no advertising for instance.

    The first thing to do if banner ads are a possibly, profile your visitors by demographics (from where (search query, engine, link, geographic, when (best time of day and day of month), and profile your site by page most entries, most accessed, most time "eyes" duration).

    Take this info and make a web site statistics page and potential banner prices (introductory prices at 50% to 75% to start maybe for the first few inqueries).

    Attractively link (and somewhat low profile) this in your site in such areas such as about us, contact us, but outside of your actual gallery.

    Start getting the word out (offline and local) in industries that these targets and number may appeal to.

    It is best to limit yourself to only a small number of banners -- for if you get too many distractions your visitation rate could decline.

    Scooter24

    10+ Year Member



     
    Msg#: 1427 posted 5:02 am on Oct 20, 2002 (gmt 0)

    What is your average graphic size? How many hits/day are you receiving?

    That depends on the day. In the past three peak days it was about 1000-1100 visitors a day, each visiting an average of 7 pages. One image is an average of 150KB, i.e. the average visitor seeing 7 pages generated about 1-1.1 MB of traffic.
    Under normal conditions webalizer registers about 700-800 visitors/ day and around 5000 page views. But there are about 4% of visitors who generate almost 50% of the traffic (people who view between 50 and 500 pages).

    The first thing to do if banner ads are a possibly, profile your visitors by demographics

    Are there any tools which help doing that? Webalizer isn't too detailed - concerning the country information it can't tell me where 80+% of the people come from. Despite the fact that I get over 600 search engine hits/day, the top 20 search terms only cover a total of 1000 hits in the past 19 days.

    Attractively link (and somewhat low profile) this in your site in such areas such as about us, contact us, but outside of your actual gallery.

    But these pages get very few hits. Should I sell the banner according to a click though rate, revenue would probably be minimal. On the other hand, putting a top banner on the image pages (those which get the bulk of the traffic) would somehow spoil the appearance. Besides traffic - at 700-800 visitors a day - isn't too huge, although it might be very focused. Are there any potential advertisers which would pay a fixed fee/month or a fee according to the page view (and not to the click-through)?

    diddlydazz

    WebmasterWorld Senior Member 10+ Year Member



     
    Msg#: 1427 posted 11:26 am on Oct 20, 2002 (gmt 0)

    Hi Scooter

    <-- One image is an average of 150KB

    That seems extremely large ! If they are just photos then you could get them down to at least 40-50kb without losing any quality.

    In fact you could get them as low as 25k if not lower

    Dazz

    Scooter24

    10+ Year Member



     
    Msg#: 1427 posted 1:31 pm on Oct 20, 2002 (gmt 0)

    <-- One image is an average of 150KB

    That seems extremely large ! If they are just photos then you could get them down to at least 40-50kb without losing any quality.

    In fact you could get them as low as 25k if not lower

    That depends on the quality. My images are 800x600 and are top quality - you can't see any JPEG artifacts. I've tried saving an image in a lower quality setting. The size of this specific file went down from 166K to 125K, but you started seeing a blockiness in the sky.
    At the moment I'm playing with the idea of selling the images to magazines (as a stock photographer), and that's why I'm sticking with this quality level - as I said, at the moment. Shouldn't this work out, I might just resize all images to 640x480, increase compression and try to cover my expenses with advertising.

    diddlydazz

    WebmasterWorld Senior Member 10+ Year Member



     
    Msg#: 1427 posted 1:44 pm on Oct 20, 2002 (gmt 0)

    Yes, if quality is vital then I see what you mean.

    There are some graphic gurus here that may be able to help, you might try posting something in the graphics forum I am sure they might be able to help you get the file size down, for instance there is a way of just compressing certain parts of the photo that won't affect the quality.

    good luck

    Dazz

    Woz

    WebmasterWorld Senior Member woz us a WebmasterWorld Top Contributor of All Time 10+ Year Member



     
    Msg#: 1427 posted 1:56 pm on Oct 20, 2002 (gmt 0)

    If you are thinking of selling stock photos to magazines, what about having only medium quality copies for viewing online with a members/buyers area where clients can download the real thing. That would keep the download usage down somewhat.

    Onya
    Woz

    cminblues

    10+ Year Member



     
    Msg#: 1427 posted 2:47 pm on Oct 20, 2002 (gmt 0)

    Are you sure that nobody is hard-linking your images from another site[s]?

    If you want to get rid of this, and you have Apache, httpd.conf or .htaccess customization is enough.

    Here some info:
    -> In Google, try:
    apache htaccess hard linking images
    [without quotes of course]

    cminblues

    mykel

    10+ Year Member



     
    Msg#: 1427 posted 3:06 pm on Oct 20, 2002 (gmt 0)

    Despite the fact that I get over 600 search engine hits/day, the top 20 search terms only cover a total of 1000 hits in the past 19 days.

    You can easily get webalizer to show all search strings, not just the top ones. You have to add the line
    AllSearchStr yes

    to your webalizer config file, or get the provider to do it if you don't have access. This will place a link "all search strings" under the top 20.

    europeforvisitors



     
    Msg#: 1427 posted 4:44 pm on Oct 20, 2002 (gmt 0)

    You'd actually be doing most of your visitors a favor by reducing image size to something reasonable (e.g., a width of 400 pixels or less and 50% JPEG compression). Display times would be much faster, especially for users who don't have broadband connections.

    Rincewind

    10+ Year Member



     
    Msg#: 1427 posted 5:07 pm on Oct 20, 2002 (gmt 0)

    You need to do some better image handling here. I run a gallery site for my own stock photography. I don't get the traffic you do but these stats should scale up. I get about the same visits per month that you get each day. I use less than 0.5Gb bandwidth per month so scale up to your site that would be 15Gb.

    I use three levels of images. First a small thumbnail 150px across and less than 5Kb (maximum compression). Second an enlarged image that displays within the normal site, size 450px and between 20-30Kb(minimal compression). Finaly for those few people who want to see it full sized a full screen 800px or larger image with no compression at all.

    Adding in the intermediate file size should reduce your bandwidth significantly.

    zeus

    WebmasterWorld Senior Member zeus us a WebmasterWorld Top Contributor of All Time 10+ Year Member



     
    Msg#: 1427 posted 8:10 pm on Oct 20, 2002 (gmt 0)

    Even inserted code which automatically adds a deny from xx.xx.xx.xx line to .htaccess when somebody tries to download the site.

    That sounds interesting how?

    I think we are some what in the same categorie I got a huge page with pictures and about 5000 visits a day, thevisitor are also going through a few sites.

    My ave. pictures are 80-100kb, the best graphic tool is Photoshop 7.

    zeus

    musicales

    10+ Year Member



     
    Msg#: 1427 posted 8:37 pm on Oct 20, 2002 (gmt 0)

    I ended up with bandwidth problems on a site that exceeded 60GB a month. In the end I limited certain areas of the site to subscribers for a small fee. I also put in cookies so that non-subscribers were limited to three downloads a day. That way most of the content remained free but was easier to use if you paid up. Obviously you can get round the cookie easy enough, but most won't and didn't. I ended up with some nice income from the subscriptions and bandwidth use fell to 35GB (though page views to the site actually increased).

    Scooter24

    10+ Year Member



     
    Msg#: 1427 posted 9:07 pm on Oct 20, 2002 (gmt 0)

    Are you sure that nobody is hard-linking your images from another site[s]?

    Yes - I've already installed the anti-leeching code. Besides it would show up in the access logs.

    shady

    10+ Year Member



     
    Msg#: 1427 posted 9:22 pm on Oct 20, 2002 (gmt 0)

    As rinsewind states, you would definitely save bandwidth by making three graphic sizes instead of two. 450px is big enough to give a good quality view of a graphic.

    I guess this may be a lot of work for you! There are batch resize tools out there which would take some of the burden.

    Rincewind

    10+ Year Member



     
    Msg#: 1427 posted 9:31 pm on Oct 20, 2002 (gmt 0)

    Shady - I use a php gallery script to sort out all the images. I ftp up the large images and run the gallery's "add new pictures" option and it creates all the thumbs and web pages and database entries. Makes adding new stuff a dream.

    Scooter24

    10+ Year Member



     
    Msg#: 1427 posted 10:04 pm on Oct 20, 2002 (gmt 0)

    I ended up with bandwidth problems on a site that exceeded 60GB a month. In the end I limited certain areas of the site to subscribers for a small fee.

    Musicales,

    what kind of content were you offering and why would people pay for it?

    Scooter24

    10+ Year Member



     
    Msg#: 1427 posted 10:13 pm on Oct 20, 2002 (gmt 0)

    you would definitely save bandwidth by making three graphic sizes instead of two. 450px is big enough to give a good quality view of a graphic.

    I guess this may be a lot of work for you! There are batch resize tools out there which would take some of the burden.

    The problem is not resizing the images - a batch job will do that overnight. The big problem is generating the html code to add that third layer of detail (first layer the thumbnails, second the medium size images, third layer the big images). Try doing that for a site with 2100 images.

    Still, I agree that it's a good idea to offer the medium and large size as an option to visitors. I saw a photo gallery where in the thumbnails at the lower left and lower right corners there were small buttons '640x480' and '1024x768'. Quite a cool solution, but I'd need the right software for that (generation of the html pages).

    shady

    10+ Year Member



     
    Msg#: 1427 posted 10:26 pm on Oct 20, 2002 (gmt 0)

    Perhaps for a short term solution you would be better to simply resize the large image.

    I am assuming your pages are handwritten html, in which case it is a long task. If they are all the same format, you may be able to have a programmer write a program which would perform a global update on your html documents - its probably not the worlds most difficult techie task!

    fathom

    WebmasterWorld Senior Member fathom us a WebmasterWorld Top Contributor of All Time 10+ Year Member



     
    Msg#: 1427 posted 10:26 pm on Oct 20, 2002 (gmt 0)

    The first thing to do if banner ads are a possibly, profile your visitors by demographics ...

    Are there any tools which help doing that? Webalizer isn't too detailed - concerning the country information it can't tell me where 80+% of the people come from. Despite the fact that I get over 600 search engine hits/day, the top 20 search terms only cover a total of 1000 hits in the past 19 days.

    Many visitors will likely come in under their host ISP and many of these tend to be dot.com and dot.net (or extremely hard to identify using this approach. (language of browser may help).

    Attractively link (and somewhat low profile) this in your site in such areas such as about us, contact us, but outside of your actual gallery...

    But these pages get very few hits. Should I sell the banner according to a click though rate, revenue would probably be minimal.

    You do not want to advertise banner space on your high traffic pages. The visitors likely going here are generally not the same ones looking to advertise. As I said before, you may want to start locally (off-line).

    On the other hand, putting a top banner on the image pages (those which get the bulk of the traffic) would somehow spoil the appearance. Besides traffic - at 700-800 visitors a day - isn't too huge, although it might be very focused. Are there any potential advertisers which would pay a fixed fee/month or a fee according to the page view (and not to the click-through)?

    Your primary pages are images/gallery stuff - a banner can look the same (in context, theme and appeal).

    As far a pricing model, start low -- the idea is to have these fees paying some of your cost -- and not necessarily to creates profit revenue. Your web site is not a portal with enormous traffic under a variety of topics but it can help reduce your associated cost.

    Gorufu

    10+ Year Member



     
    Msg#: 1427 posted 1:07 am on Oct 21, 2002 (gmt 0)

    From msg #:7
    I have disabled hot linking (with .htaccess), disabled directory browsing, banned most known download agents. Even inserted code which automatically adds a deny from xx.xx.xx.xx line to .htaccess when somebody tries to download the site.

    Are you using mod_rewrite to block hotlinking and download agents?

    Most download agents can be configured to fake MSIE or Netscape User-Agents. They usually have no referring URL and can be easily spotted in your access_log

    For example
    "GET /image.jpg HTTP/1.1" 200 151204 "-"
    "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)"

    The following code in mod_rewrite will block hotlinking and allow access to normal browser User-Agents even if there is no refering URL.

    RewriteEngine on
    RewriteCond %{HTTP_REFERER} !^$
    RewriteCond %{HTTP_REFERER} !^http://domain.com [NC]
    RewriteCond %{HTTP_REFERER} !^http://www.domain.com [NC]
    RewriteRule .*\.(gif¦jpg¦png)$ - [F,NC]

    RewriteCond %{HTTP_REFERER} !^$
    The above line allows access without a refering URL. Removing it will prevent downloading of images for most bots.

    However Proxy servers will also be blocked. ISP's also want to reduce their bandwidth costs. Forunately most Proxy servers use Mozilla/3.01 (compatible;)

    IMHO it is not a good idea to block Proxy servers. This is what I use to block hotlinking and most browser download bots. Proxy servers are allowed access.

    RewriteEngine on
    RewriteCond %{HTTP_USER_AGENT} !^Mozilla/3.01
    RewriteCond %{HTTP_REFERER} !^http://domain\.com [NC]
    RewriteCond %{HTTP_REFERER} !^http://www\.domain\.com [NC]
    RewriteRule .*\.(gif¦jpg¦png)$ - [F,NC]

    From msg #:10
    Under normal conditions webalizer registers about 700-800 visitors/ day and around 5000 page views. But there are about 4% of visitors who generate almost 50% of the traffic (people who view between 50 and 500 pages).

    If 4% of visitors are generating 50% of the traffic you may have a big problem with browser download bots faking normal visitors.

    keyplyr

    WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



     
    Msg#: 1427 posted 5:54 am on Oct 21, 2002 (gmt 0)

    RE: msg #:7

    Thank you for posting that Gorufu. I am using the standard RewriteCond code to stop requests from other than my domain, along with a long list of UA disallows.

    You say ...most Proxy servers use Mozilla/3.01 (compatible;)... This is what I use to block hotlinking and most browser download bots. Proxy servers are allowed access.

    How about international and other portals/proxy cache servers who do not use Mozilla/3.01 (compatible;) as their UA? While I understand this code accomplishes more efficient screening for UA spoofing, I still am hesitant to use an accross the board block with unknown variables when I get significant international referrals.

    musicales

    10+ Year Member



     
    Msg#: 1427 posted 6:54 am on Oct 21, 2002 (gmt 0)

    Scooter24 - that would be telling!

    This 52 message thread spans 2 pages: 52 ( [1] 2 > >
    Global Options:
     top home search open messages active posts  
     

    Home / Forums Index / WebmasterWorld / Webmaster General
    rss feed

    All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
    Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
    WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
    © Webmaster World 1996-2014 all rights reserved