How much server do I need for a tile server?

Dedicated? Cloud?

[Sierra_Dad]

For an exciting business opportunity, I am just trying to find a rough idea of how much server I need. I have little experience in server hardware, so I at least want to figure out the number of zeros.

Suppose I want to run a tile server like the one that powers Google Maps. OK, maybe not quite that big, but close.

Let's say further that I have pre rendered all the tiles. I now have 35 million png files of about 15K each, so I have maybe 500 Gig of files.

These files fit into a nice address structure like.

/tiles/15/1634/2178.png

So essentially all the server is doing is serving up files. Simple, right?

Now lets add 30,000 users that will be accessing these tiles. These are not casual users that are merely browsing a webmap on a website. These are serious hardcore users using software to download these tiles in bulk. They may download 1 GB at a time, which could take them several hours.

I imagine that puts some load on the server.

$$
I think I can eliminate shared hosting, even if they promise unlimited storage space and unlimited bandwidth.

$$$
I'm wondering if I can get by with a dedicated server. Can that keep up with the load?

$$$$
But will the dedicated server be enough? Should I go for a scaleable cloud solution with the whole redundancy, replication and all that? I'm still vague on how much cloud hosting costs because they can't tell you till afterwards. (Oops, last month cost you a million dollars.)

Once we have the four dollar signs, I get to where I need to charge people a subscription model to make this work.

Thanks for any insight.

[Leosghost]

IIWY I'd talk to some of the pubcon sponsor hosting companies directly about this..

[g1smd]

Something on the scale of Google Maps needs at least tens of thousands of servers.

Finger in the air maths: Say you have 30 000 users and they each pull 10 GB in a month.

You're looking at 300 TB bandwidth, and that's likely to cost well in excess of USD 50 K per month.

You'd also need to factor in the costs of billing/collecting money from 30 000 people; and for having a CSD for dealing with failed payments, refunds and cancellations.

Add set-up costs and other expenses and you'd be on to an instant loser if you had start up capital of less than a million (to keep you in business for a year or so).

[lammert]

It is not the server CPU power or hard disk capacity which is the limit, but the bandwidth and amount of RAM per server. 500 GB of files can be easily served from a handful of servers which in total have enough RAM to cache all files. With RAM disks or full RAM caching you will be able to serve 1000 or more files per second per server, provided that your bandwidth allows it. I have currently a small dedicated server serving a sustained stream of 400 requests per second 24 hours per day and the CPU and disks are almost idle.

If we take the 300000GB per month which g1smd estimates, that comes down to 10000GB per day or about 1GBit/sec. With files of an average size of 15 kbyte you will have around 8000 file requests per second. You can use 10 servers with a large amount of RAM to serve it hardware wise, each serving around 100 Mbit/sec. That is nothing special and done by many file servers in office environments on a regular base.

[Sierra_Dad]

Something on the scale of Google Maps needs at least tens of thousands of servers.

Well, I can't promise the same popularity of Google Maps. 500 million users is greater than 30,000

Finger in the air maths: Say you have 30 000 users and they each pull 10 GB in a month.

You're looking at 300 TB bandwidth, and that's likely to cost well in excess of USD 50 K per month.

That may be in the ballpark. More likely, they are pulling 10 GB in a month, and that is 30 TB a month.
One of the PubCon Exhibitors offers a 30TB dedicated server plan at $500/month, but that assumes one server is enough.

Add set-up costs and other expenses and you'd be on to an instant loser if you had start up capital of less than a million (to keep you in business for a year or so).

Well, I wondered why I've never pursued this. I've just always assumed it was a really big number.

No, I don't have a million in startup capital. I do have the customers, though. If I could distill this to a per person number, I can make it a prepaid offer.

But plan B is looking better all the time.

[Sierra_Dad]

More likely, they are pulling 10 GB in a month, and that is 30 TB a month.

I meant to say they are pulling 10 GB in a year, and that is 30 TB a month.

[g1smd]

Ah, that factor of ten lower might bring it into the realms of reality...

[Sierra_Dad]

OK, so I think that puts the cost at somewhere between $1000 and $5000 per month. Not cheap, but less expensive than hiring a developer in the US.

[damon_cool]

Try amazon cloud server, you can set one up and try it on a small scale before you go full scale. You can then ratchet up the specs of the machine to get the performance you need. You might have to spend a chunk of change to test it out, be you'll know what you need before committing to a big purchase of hardware or hosting contracts. You can provision a new machine and have access in less than 30 minutes. If you're serious and don't know what you need you could spend a few hundred on a proof of concept at AWS, potentially saving thousands in the long run. The nice thing is you only pay for what you use and you can turn it off at anytime.