homepage Welcome to WebmasterWorld Guest from 54.227.12.219
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / WebmasterWorld / New To Web Development
Forum Library, Charter, Moderators: brotherhood of lan & mack

New To Web Development Forum

This 31 message thread spans 2 pages: 31 ( [1] 2 > >     
Hosting from home
Guide to hosting your own server from home
mack




msg:961839
 8:32 pm on Apr 24, 2005 (gmt 0)

Over the past year or so there has been a growing trend of post from users who want to host their sites on their own systems.
I think it would be good to hear from people who have actually done or are doing this. I don't think we should just say yes or no, but offer as much practical advice as possible and voice our options on this matter.

It has to be said that in 95% of cases hosting with a web hosting company is by far the best option available. When you use a hosting co you aren't just buying disk space and bandwidth you're buying their experience and their expertise. Web hosting companies handle the complexed issues involved with server administration on a day to day basis and are generally very able to provide a high level of service and availability.

I took the plunge about a year ago and began hosting my site from home. for me the entire outcome was based on a financial decision. The project I was working on would involve using at least 2 fairly high speck dedicated servers. The cost of leasing the boxes was just to high for what is no more that a hobby project.

The ruit I decided to go down was to build the servers myself and then host them from a spare room at home. I am far from being a networking or hosting expert but I did a lot of reading (mainly here) and was able to be up and running in a matter of days. It is fairly easy to have a server available but that is pretty much the tip of the iceberg.

Technically all you need to do is build or acquire a computer that you intend to use as a server. Install the operating system of your choice and a server application such as Apache or IIS. In my case I went with a combination of Linux/Apache although it has to be said the principals for home hosting are very similar between Windows and Linux. The principles are the same, only the procedures vary.

Lets imagine we want to set up a server to serve a website that is publicly available via www.example.com Because all my self hosting experience has been on Linux servers I will stick with what I know. I am hopeful someone will be able to offer us some info about hosting on Windows based web servers.

The first thing you will need is an internet connection that is able to handle your traffic load. When hosting from home this will be your biggest obstacle. The cost of a very fast internet line will very often be a barrier. As a result of this connection speed will very often be your bottleneck.

To make an attempt at true hosting you will also require a fixed ip address. A fixed ip address is an ip address that never changes. some services provide this at an extra charge. Be sure to ask your provider if this is available. It has been knows for some providers to allocate the same ip address to users for a period of time then suddenly flush their ip configuration and allocate different ip addresses to their users.

A static ip address is one of the most important factors to take into account before you even think about hosting from your own location. Another important piece of information you need to find out from your provider is do they block ports. A port is almost like a data stream. For example the default port for web traffic (http) is port 80. If you later found out that your connection provider was to block port 80 you would not be able to serve any web pages on the default port. It is still possible to serve page an another port but it is always the preferred option to provide services on the port they where designed for.

There are various other ports that you may need. email, ftp, ssh, https to name but some. We will not go into these other service ports in any great detail here. But ideally you should look into using a connection provider that does not block any ports. You should also email any potential providers just to make sure "Am I allowed to use your service to host a web server?" if they say no then you have just saved yourself having your service terminated for abuse. It is very important to adhere to your providers terms and conditions at all times.

The second item on your wish list should be a router. I would be tempted to get a router that has a built in firewall. Today many routers act as modem/router/firewall combinations. The main benefit to this is you do not require a separate unit for each component.

You can also configure it from on admin interface. It is very important to get your router/firewall configured correctly. not only does this act as your internet access point it also acts as your first point of security.

The firewall will help to protect your network from intrusion from the outside world. Lets imagine you have a server and your pc on the same LAN. You will want users to be able to access your server via http but you will not want them to be able to access your pc under any circumstances.

There are many different routers on the market and it would be simply impossible to go into any great detail for all of them so we will try and keep it as simple and general as possible. The first step is to set up the router for internet access.

To do this you should be provided with all the required information from your provider. Once you have done this you should test your web connection to make sure it is working.
The next thing you should do it to configure your server. In my case the server was a Linux server that connected to the router using standard network cable via it's NIC (network interface card) Many Linux distros such as Suse and Redhat have graphical user interface tools to help you set this up correctly.

The important pieces of information you need to provide is the computers IP address. This is not the IP address of your internet connection it is an internal LAN ip address for example 192.168.1.2 You should set your server so use a static ip address and provide an address within the same subnet as your router. This is very important because if the server and router are on different subnets then they will not be able to see each other on the network. It also makes sense to have your LAN IP addresses sequenced in order.

The router ip address is the address you visit to view your router admin area. This is usually 192.168.1.1 if you are not sure about this ip address please consult your router manual or user guide. We now have the router on 192.168.1.1 and the server on 192.168.1.2 You will also need to provide "name server" information to your web server. A name server is used to help the server convert domain names into numeric ip addresses. you should have 2 name server addresses. typically NS1 and NS2. usually there will also be a routing option within the user interface. Within the routing section all you need to do is provide a gateway IP address. The gateway ip address will be the LAN IP address of your router. (i.e. 192.168.1.1)

You can set up your network card using either Yast (Suse) or Redhat installer. Most modern Linux distributions have similar tools. Unfortunately I can only comment on what I know and have used.

If you have a windows PC on your network I suggest you also set it up to use a static ip address. By doing this you will be able to turn of DHCP on your router and keep things nice and simple.

DHCP is a system whereby the router allocates a dynamic address to all client computer on the network. In order to configure windows to use a static ip address you should do the following.

Click on start > control panel and your windows control panel will open. Click on "network connections" this will open up a screen displaying your network connections and their current status.

choose the connection that refers to your current connection between your computer and your router. and left click it.

Choose "properties" from the drop down list. This will open the properties dialog box for your local area connection.

Click on internet protocol (TCP/IP) and click on the properties button. this will open the internet protocol (TCP/IP) dialog box.

Within the dialog box you have the option to assign an IP address or use DHCP. DHCP simply means that your router is automatically issuing dynamic IP addresses to all client computers on your local area network. Select "use the following ip address" and insert an ip address in the box provided. for example 192.168.1.3 You will also need to insert 2 values for your name servers NS1 and NS2. These are the same values we used for setting up the network card on the server.

You can then click OK and your settings should be saved.

You should now re-visit your router admin page. You now need to make sure DHCP is turned off. This not really required but is seams wrong to have an unused service running. usually you then need to specify a hostname and ip address for each client on the network. Not all routers require this although both that I have used did require this.

This is basically telling the system that this IP address is reserved for this machine. At this point make sure all ports are closed. By ensuring all ports are closed no part of your network will be open to the outside world.

Now that you have your network pretty much set up you can how begin configuring your server to host your website. For purposes of this example we will assume your name is "Joe" and you have a user account on your Linux computer called "Joe" If your PC and server are able to see each other across the network you should be able to use ftp across your LAN. In order to do this you need to have an FTP server installed on your server. Some popular examples on Linux are vsftp or proftp.

Bother are very good ftp servers and are very comparable. Most Linux distributions will come bundled with some sort of FTP server. By default it may not run as a service at boot time so you may need to set this up in your inet.d configuration.

If you have an ftp server set up and running you should be able to log into ftp://your.server.ip using your Linux system username and password. Your website files will be stored in the /public_html folder. In reality what you are actually doing is ftping into /home/Joe/public_html

If you already have a site built you can simple move the files into your users web space. If you set this up correctly you should be able to browse your site using the following address. [server.ip.address...]

It is important to remember that if web traffic was to arrive now to your server it would be pointing to the IP address of the server and not the correct location of the web files. In order to allow people to be able to view the files via a domain name we need to use what is called name based virtual hosting.

Name based virtual hosting as a fairly simple method of telling the server where to send users who arrive at the server. It used the domain name they arrived via to determine where they should be sent.

Virtual hosting can be set up using your httpd.conf file (apache configuration file) To set up a virtual host to enable the domain www.example.com to resolve to Joe's public_html directory you would use the following syntax

ALWAYS BACKUP YOUR HTTPD.CONF BEFORE MAKING ANY CHANGES TO IT!


<VirtualHost *>
ServerName www.example.com
DocumentRoot /home/Joe/public_html/
</VirtualHost>

The first line is the opening tag for the virtual host. the "*" is to let the server know that this virtual host is accepting traffic on any port. You couch replace this with a numerical value if you wanted to block non standard ports requests. On this case I am blocking non standard ports from the router so have left this as default for any port.

"ServerName www.example.com" this is to tell the server what domain it is to accept incoming traffic for. Any other domain that is not placed within a virtual host will be ignored.

"DocumentRoot /home/Joe/public_html/" this line tells the server where the files for the users doc root are located.

The virtual host is then closed almost in the same way as you would with an html tag. </VirtualHost>

In order for your virtual host to work you need to save the changes to your httpd.conf file and restart apache. If you have a control panel installed on your server you should be able to do this from within there. Alternatively, try shelling in as root and running "apachectl restart"

You should get the following output in your shell "httpd restarted" If apache was successfully restarted then you should be able to try the following test.

Go onto a pc on your network and alter the host file as follows.


server.ip.address www.example.com

If you visit www.example.com on your pc you should now be able to view your website.

This is because you are placing a request for the domain and the request is being relayed to the server. The server will receive the request and find there to be a suitable virtual host entry. It then displays the content based on the location specified within the virtual host entry. The reason I suggest adding it to your host file is because you do not want your request to go into the world outside of your network. You are just making the http request over your LAN based on a domain name.

You can have several virtual host entries. Virtual hosting is now pretty much the standard used by the hosting companies to offer shared hosting. You can have literally hundreds specified within your httpd.conf file but lets take things slowly for a start.

Here is how 2 virtual host entries would be set up.


<VirtualHost *>
ServerName www.example.com
DocumentRoot /home/Joe/public_html/
</VirtualHost>

<VirtualHost *>
ServerName www.example.net
DocumentRoot /home/Sue/public_html/
</VirtualHost>

Continued on next message.....

 

mack




msg:961840
 8:32 pm on Apr 24, 2005 (gmt 0)

Technically you now have a web server set up.

What you now need to do is think security. If you are using a mainstream distribution then you will be able to check for updates to your operating system online. For example Suse has "Suse you" and Redhat had "online update" both of these systems are designed to keep your operating system and related software up to date. This is very important from a security point of view because the updates will contain security patches and bug fixes. Ensure you keep your system as up to date as possible. No system will ever be 100% secure but you need to be prepared to keep on top of updates and security fixes.

I also find it is very useful to follow news groups and forums very carefully. Very often that can be great for giving you a "heads up" on potential threats.

You need to also pay very close attention to your router. The router also has software that is usually updated periodically. From a security and reliability point of view it is important to ensure you use an up to date version of their firmware. The router is the first point of contact between your network and the internet. You need this part of the system very up to date. when you are sure your server and your router are up to date you can now begin the process of opening up your server to the internet.

At this stage I recommend you pause and have a proper read through your router handbook. I cant offer any advice relating to any specific router. All I can do is explain the procedure and you will need to work this out based on your own equipment.

The first think you need to do is locate a feature called "port forwarding" (or something similar) What you then need to do is set up a port forwarding rule to ensure that all port 80 traffic is forwarded to the ip address of your server. There is no easy way to explain how to do this because every router admin interface will be slightly different. Some routers also have pre-set rules that you can apply to a given ip address. For example Dlink routers have a preset rule for a "web server" All you need to do in this case is to specify a LAN ip address where you want web traffic to be forwarded. At this point is is very very important to make sure that ports that you do not need are closed.

Currently all we need is port 80. You can close other ports. If you have a domain that is registered with a domain registration company or a web host you will need to make changed to the DNS in order for the domain to point to you. What you need to do is use what is called an A entry to point the domain (www.example.com) to your external IP address. The external ip address is the IP that can be used to reach your network from the web. This is sometimes referred to as a Wan address.

Usually this can be set from within your host or domain registration companies control panel. If your host makes the changes take effect right away then you should be able to then go to www.example.com and see your website.

This is the part that is difficult to explain.

Of you are on your network you will not usual be able to see your website without setting an entry in your pc's host file. The router is designed to handle requests from the outside. So even if you type in www.example.com the chances are it will display your routers admin area as opposed to your website. For this reason I suggest you try viewing your web pages from outside of your network to ensure it is functioning correctly.

An alternative is to connect through a proxy server to try and view your pages that way. The problem with using a proxy server is they can be very slow and as a result it will not give you a fair indication f the speed of your pages. The best thing is either do use a second home internet connection (if available) or use a friend or family's internet to make checks.

Basically you now have a working web server hosting your content from your own location but you are lacking in a lot of functionality such as statistics, logging, cgi-scripts etc.

What we will now do is add a few features to enable you to get a bit more from your server.

We are going to set up your server so that it will store log files in a file called "access.log" What I tend to do is store my log files in a folder called /logs just above the users home directory. So in the case of Joe his logs will be located at /home/Joe/logs

The first thing you will want to do is create a directory for storing the logs and make it writeable. log in to your server with shell and do the following...


cd /home/joe
mkdir logs
chmod 777 logs

You have now created a directory to store the users log files (access.log) we now need to configure apache to write them.

What you will need to do is open your httpd.conf file again and make an edit to the virtual host


<VirtualHost *>
ServerName www.example.com
DocumentRoot /home/Joe/public_html/
CustomLog /home/Joe/logs/access.log combined
</VirtualHost>

You will notice we have added a new line "CustomLog /home/Joe/logs/access.log combined"

This simply tells Apache where to store the logs and what to call the log file. What you can do now is download the log file from the server to check the logs on a pc based web analyser system or you can install a script on the server to do this in real time. Examples of those scripts are AWstats and Weblizer. We will not go into a lot of details about installing either of those because they each have a very good read me file and the install is fairly easy so long as you know your way about the operating system and server environments.

One very important piece of information that you need to set before you actively start logging your real users is the server time and system clock. log in as root and type "clock" to make sure this is correct. If it isn't then your logging will be way out of sync.

If you wish to use cgi scripts on your server you firstly need to have the Perl scripting language installed. You can determine this by running shell and typing "whereis perl" this should return the locution of the Perl interceptor.

If your shell output displays a location then perl is installed, if not then you will need to install perl either using the installer than came with your Linux distribution of by downloading the latest version from the web. We will not go into this in any detail here because this is an entire discussion in it's self.

The next stage is to determine where cgi script should be allowed to run. The normal location is /cgi-bin we will use this for the purpose of this example. The first thing to do is create your cgi-bin directory above public_html. You can do this using ftp or shell. If you use shell you can do the following


cd /home/Joe/public_html
mkdir cgi-bin
chmod 757 cgi-bin

You now need to make a further modification to your httpd.conf file.


<VirtualHost *>
ServerName www.example.com
DocumentRoot /home/Joe/public_html/
ScriptAlias /cgi-bin/ /home/Joe/public_html/cgi-bin/
CustomLog /home/Joe/logs/access.log combined
</VirtualHost>

What we have done is added the following line to our virtual host "ScriptAlias /cgi-bin/ /home/Joe/public_html/cgi-bin/" what this does it inform the server that the directory /home/Joe/public_html/cgi-bin/ is allowed to run cgi script.

Be sure to save the file on close and restart apache. You need to restart apache every time you make a change to your httpd.conf file or the changed will not take effect. Changes only take effect after you "start" or "restart" apache.
Technically you now have a web server set up and running and are your own host. but this is not the end of the road.

You are now responsible for all server updates, problem diagnosis, security auditing, hardware maintenance to name but a some of the items that will now become your responsibility. The hard truth is if the server goes down the buck stops with you. You need to be able to get the server back up and running ASAP. Here are a couple of worst case scenarios.

.You go to bet at 11 each night and wake up at 7 your server goes offline at 11.30pm

.You have a hard disk failure

.Your connection goes down

.You have a power cut

There is a simple solution to all of the above. It's called using a web hosting company. Running your own server is the easiest way to get yourself seriously stressed out. If you're still keen then here are a few pointers that might be enough to get you through.

if your server goes down when you're either away or asleep then there really isn't a lot you can do. One idea is to educate a willing friend how to bring your server back online using secure shell. Ask then to check up on your site at various times when they go online and if they discover a problem they can try to resolve it. This would be an even better agreement if the person had server experience. If not then you may be able to educate them enough to carry out simple tasks.

A hard disk failure or other hardware failure can be literally disastrous. If this happens it is vital they you have a backup that is not located on the server that has failed. That way when you have the system repaired and ready to roll you can place your data back into place. This sort of failure will almost always lead to a large period of downtime. In reality expect a hard disk to have you down for the best part of a day. it's not just a case of swapping a disk it's almost a full set up and re-configuration. You don't just loose your date you loose all your configurations and settings etc. It may be worth your while investigating a full partition backup. this sort of application can involve quite a large initial investment but long term it may add up to a substantial saving.

If your internet connection goes down then you have a real problem. The obvious solution is a backup line. the problem being this will effectively double your connection price. You will also need to have a system in place that will alter your dns when your primary line goes down.

Power cuts are a serous problem for anyone hosting from home. The most you can realistically offer is a UPS system. A UPS is like a bank of batteries that goes between your power source and your computers. If the power source is lost then the batteries will continue to power the systems. Batteries will not however be able to keep your systems alive for very long. You're talking half an hour tops for a high quality, home style UPS. If you want to pay top dollar you could probably get a more robust solution with extended outage protection. Most UPS systems will also have a feature that is designed to power down the computers naturally as opposed to just powering them down when the power runs dry. This can help to prevent file corruption or operating system damage that can result from forced shutdowns. The next step up from a UPS is a second electrical source. Usually this would be in the form of an electrical generator, but lets be realistic. Can you imagine how your neighbours would react if a petrol generator was to kick into life at 3am on a Monday morning? Your 404 pages would be the least of your worries. For the average home hoster a UPS is about as far as you can realistically go. One tip that may be helpful to get you back up and running ASAP is to enable "power on LAN" within your bios. This is a procedure that allows a computer to turn it's self on as soon as the router received a power supply. It works by detecting it at the LAN card. If you have all your required services set to start at boot time then everything should revert to working order when the power comes back on.

so is home hosting worth it?
From a learning experience I would say yes. From a practicality point of view I would say no.

There are some things that really are best left to the experts. I would say hosting from home has to be a final option when all others have been exhausted. I like having my machines hosted in my house but to be honest there is little or nothing I can do how that I couldn't do it they where in a datacenter somewhere.

Collocation appears to be a real alternative to home hosting.
Hosting from home can be a great learning experience but there are more realistic options available.

Mack.

encyclo




msg:961841
 12:55 am on Apr 25, 2005 (gmt 0)

Another stunning introduction article mack! I think that self-hosting is every webmaster's dream, but as you have indicated it is not always an easy choice. I agree that getting a reliable connection is about the most important barrier, and it is where you have the least control.

I would strongly recommend anyone considering self-hosting to familiarize themselves with running a server by building a test setup first, where connections and downtime are not an issue. I prefer Linux hosting too, and it is best to pick a Linux distribution and stick to it: whether you choose Mandriva (formerly Mandrake), Fedora, Suse or Debian, or also one of the BSDs, they are all up to the task of web hosting. Get to know how your distribution works in detail, including managing services, adding components and updating routines.

Running a web server can be fun, and it would be a great feeling to have full control over your sites with them all running on a machine sitting in your office - but beware of the downsides: the fun fades away at 2am during a power cut or when trying to recover from hardware failure to get your sites back online. Suffice to say, I run only test servers at home, and always host at a datacenter.

mack




msg:961842
 5:00 pm on Apr 25, 2005 (gmt 0)

I would agree, hosting your boxes in a data centre is the way to go. Currently I am looking into moving my servers out of the house and into a colo.

I will however keep a server at home with the exact same set up as my remote boxes. That will make it nice and easy to test applications as well as testing updates before I put them on the live servers.

Mack.

lexipixel




msg:961843
 1:19 am on Apr 27, 2005 (gmt 0)

When I started migrating from the BBS world to the web, I pulled a full T1 into my house and began offering dial-up internet service, web hosting, and co-located servers for a few other local sysops that didn't want to take the financial plunge.

At the time, (around 1994-1995), a T1 cost me roughly $1500.00 month... as time went by, I realized I wasn't coming anywhere close to using the bandwidth I was paying for and cut back to a fractional T1, (roughly 384k). That cut the cost to about $700/mo.

While it was great having the servers sitting on a rack in my own office, what was lacking was redundancy... when the pipe went down EVERYTHING was down.... I was using @HOME Network, and they were building out their infrastructure --- it wasn't uncommon for an outage 5 states away to put everyone connected to my small ISP business offline for 1/2 a day... and it was me who had to answer the phone 7/24 to explain why dial-up constomers couldn't connect and why sites were down.

Luckily, I got a job offer from another local ISP who was building out a state of the art data center, and part of my deal was I could bring all my hardware into their facility and run on thier redundant systems.

I stopped offering dial-up when the cable companies began wiring my town for fibre-cable access... that killed 50% of the revenue that was offsetting the frac-T1 costs, (plus I saved on all the dial-pu incoming lines I was running --- BBSing was dying and dial-up along with it).

I have never regretted NOT having the servers in the house since, and every time I even think about putting up a box, I remember the nights, weekends and vacations spent troubleshooting hardware, answering customer phone calls and generally babysitting a pile of copper and silicone instead of having a life.

Nowdays I pay for top shelf hosting with dedicated 100Mb ports, 2/47 live support and don't regret it one bit.

Even if you spend $25/mo for good "basic hosting", you would need to host 10-20 sites on your own server(s) in house just to come close to the monthly cost.

Now the flip side: Yes, running a box a home gives you immediate access to hardware, server config, and is a developers dream --- I used to code on the server, save a file and see it immediately on a second machine the way the rest of the world sees it. There is a problem though.... You get used to that type of access, then when you really need to work remotely, you inadvertantly run into something that requires a hardware or low level system reboot --- and if you aren't home, you're stuck.

What I have played around with is using a server at home for development. For a while I had a service that provided DNS so my domain name would resolve, (a third party service that acts connects the DHCP dynamic address dished out by the phone/cable company
and mates it to a static address on their nameserver)... This used to cost about $5/mo and gave me the luxury of a dedicated IP and the low cost of residential IP connectivity.

I have not hosted a machine at home for almost 4 years now and don't miss it much... although the heat from all the servers and BBS boxes did keep the office nice and toasty in the winter...<grin>.

mack




msg:961844
 2:12 am on Apr 27, 2005 (gmt 0)

although the heat from all the servers and BBS boxes did keep the office nice and toasty in the winter...

I hear ya. The room I use as my "datacenter" is a small room, about 5ft wide and about 12 ft long. It has a conventional rack that you would buy from a diy or hardware store. I adjusted the shelve heights to acomodate a pc style computer case. so i have 2 boxes top and 2 cased bottom. There is than a smaller gap to the bottom shelve where my routers and network stuff is located, That is my datacenter :) It's quite amazing just how warm it can get in there. I installed a conventional household fan pointing up behind the servers to carry as much of the hot air away.

Mack.

shri




msg:961845
 2:50 am on Apr 27, 2005 (gmt 0)

Mack, Great posts and brilliant advice.

One thing that makes a difference for our server at home (primarily used for email and a couple of small personal sites) is having a management piece like webmin ( [webmin.com...] installed on it.

FourDegreez




msg:961846
 2:53 am on Apr 27, 2005 (gmt 0)

I host some of my sites from home. It's a challenge. My worst down-time was when the JVM crashed (my sites are java-based) while I was on vacation two days before I got back. Not cool! But overall it's a neat experience.

I installed a conventional household fan pointing up behind the servers to carry as much of the hot air away.

mack, I have similar room dimensions and when the temp's outside get into the 70s and above, I turn on the little air conditioner I bought specifically for the purpose of keeping the server cool... despite the fact that it explicitly says in its warrantee not to depend on it to keep critical equipment cool =D

Ahh, the risks we take. I do have a nice UPS that has come in handy. Seeing a utility truck parked outside my house doesn't make me as nervous as before...the UPS has kept the server up for at least half an hour once while some work was being done on the line -- no outage, now that's a cool feeling. ;)

mack




msg:961847
 3:49 am on Apr 27, 2005 (gmt 0)

Yep a decent UPS is a must. Even if you just have one that stays alive long enough to power down your computers properly.

I also swear by webmin. It has several really nice features. One thing I like is being able to log in over the web and log into any server on the network, even of they do not have an external connection. Just scan for webmin servers and register them with your main webmin control panel.

Mack.

trader




msg:961848
 6:15 am on Apr 27, 2005 (gmt 0)

WoW, I can't belive how detailed and excellent your instructions are Mack. Thanks, been looking for that kind of instructions for ages. You sure delivered.

George Cooper




msg:961849
 6:48 am on Apr 27, 2005 (gmt 0)

This doesn't mention any costing. It would be good to see why this decision was financially influenced and how you came to your numbers.

You can get dedicated boxes for under US$100/month these days, including 800GB of bandwidth on multiple high speed links and a 2.4ghz processor with 1gb ram. Not to mention a service level agreement, redundant routers/links, firewall and power backup. It even spares you the cost/hassle of hardware failure, as it's their problem as they own the hardware.

Also, hosting from home would severely limit your outbound connection speed as most 'high speed' residential internet connections have relatively low speed 'outbound' speeds. For example 512k ADSL only has 128K outbound which would be fine for a small web sites but probably not anything getting more than 50,000 hits a day.

Anyway, that's just my two cents

mack




msg:961850
 6:58 am on Apr 27, 2005 (gmt 0)

With reguards to the costing descision the servers I needed where fairly high specification. minimum I needed was 2 servers with 2 gig ram a piece and 100 gig hd's. Currently running 3000ghz cpu (AMD) but Im still no where near the specification I would like.

Even a couple of hundred buccks a month is a lot to throw at a hoby site with no return.

If I was doing a project like this, and it was to be commercial then I would go with a colocaion or dedicated server provider.

Mack.

George Cooper




msg:961851
 7:07 am on Apr 27, 2005 (gmt 0)

What kind of site is it? Does the site generate any revenue via memberships or advertising? What kind of link do you have at home? If the site requires 2 such machines you must be doing a pretty serious amount of traffic.

jenkers




msg:961852
 8:10 am on Apr 27, 2005 (gmt 0)

hi,
thought I'd add my 2 pence. I'm doing a very similar thing to Mack.

The problem with a dedicated server is that to get the high spec that you want (if you want a high-spec server) with bandwidth and disk (at least in the UK) is still pretty expensive. If you need 2 or more servers then the costs mount up.

In my case I have 2 servers running full time in my garage running off a 2MB unlimited ADSL, but at the moment I need 3/4 terabyte diskspace (and this is growing).

30 - 50 per month takes care of all that - if I used dedicated servers then my costs would be ten to twentyfold.

Colocation is the only serious alternative for me - I can scrape by until I get the traffic that necessitates a bigger uplink.

But even then there is SDSL (coming down in price)bonded ADSL (not so expensive and technically getting easier and easier) and some providers are increasing their upload speeds.

Namaste




msg:961853
 11:37 am on Apr 27, 2005 (gmt 0)

we have 12 dedicated servers in datacentre. We also have 4 "production" servers in our office.

The essential differance between the servers with the datacentre and our office, is that we need to provide the infrastructure for the office servers:

- air conditioning
- UPS
- firewall
- multiple isps
- failover & load balancer
- secure area
- spare parts
- wiring

our math is thus showing us that it makes sense to undertake self hosting once you cross 30 servers.

we are of the opinion that self-hosting beyond 30 servers is definetly the way to go. There is a huge cost benefit, and a quality benefit.

I expect that with bandwidth prices falling further over the next 2 years, ad quality going up as well, a lot web businesses will take up self hosting.

Sharper




msg:961854
 4:47 pm on Apr 27, 2005 (gmt 0)

One thing touched on without a solution was monitoring.

There are plenty of inexpensive services out there for web monitoring. If you have a friend or server on another ISP, you can also setup some simple monitoring using a cronjob or open source monitoring tools like Big Brother.

If you are running a web server, the most important thing would be to periodically check that it's returning pages with the right information. You can get much more detailed than that, but you can also infer that if your internet connection is down, the page check from outside will also fail.

As for notification, set things up so that your monitoring emails you details and also emails a short message to your mobile phone. That way when it goes down at 3 am, or when you are out getting a hair cut, you know about it because your phone beeps.

The problem still remains if you travel in areas without phone coverage. At that point, if you care enough, you need to make alternate arrangements. Working a monitoring deal with someone else who monitors servers can be useful, because not only can you monitor each other's stuff, but you can work on problems when the other person is too remote.

Cheeser




msg:961855
 5:44 pm on Apr 27, 2005 (gmt 0)

Great post covering the technical aspects of setting up a site.

On the practical side, I feel compelled to share my personal experience hosting a site from home that has grown to over 500,000 unique visitors monthly with 5+ million pageviews.

It's a Harry Potter fan site I started in 2002 and originally hosted on a 800MHz server and ADSL connection for $40/mo. I had dreamed of running my own site on my own equipment for a long time and finally did it.

Very quickly, however, it became apparent that both the server and connection would need to be upgraded. I got a SDSL 1.632Mbps which has lasted about a year. I've gone through three self-built servers and just bought a brand new Dell PowerEdge 2850 and am now making the move to a colocation center. The main catalyst for moving was bandwidth and power stability. Datacenters are simply the way to go when you start needing reliability of power and bandwidth (which all equals uptime).

In the grand scheme of things, this was a good path for me to follow personally, but if you have the money up front to buy or lease a server in a datacenter, I highly recommend it. Hosting the site on my own gave me a little more security in knowing I could work on the server right in my basement if anything went wrong, but now that I'm confident of the server's reliability itself, it's a matter of making sure there's reliable power (no outages) and bandwidth (I run at nearly 100% capacity of the SDSL at all times now).

P.S. Speaking of monitoring - YES. Definitely buy a ping service or something; it has made all the difference in the world. I don't know if I can name names in this forum, but I use SecuritySpace.net for both my managed DNS and network monitoring and I get paged on my mobile phone within 5 minutes of my site going down.

Namaste




msg:961856
 6:13 pm on Apr 27, 2005 (gmt 0)

you would need a monitoring service even if you hosted with a datacentre...no differances there. Lets focus on what is needed to be done different from hosting with a datacentre.

I would like to add a word about powers. In most homes and offices we don't realise that the meter provided is of a limited capacity. If you were too add several servers, and the infrastructure to go with it, you would need to ask the power company to upgrade your meter. And do it in advance, cause they can take a few days.

Regarding name servers, it is best to use the domain registrars name servers.

shortbus1662




msg:961857
 6:28 pm on Apr 27, 2005 (gmt 0)

okay, let me ask you guys something.

I have a Dell 2650 Dual 2.6G processors, 5 36G hard drives, 1.5G of Ram, RAID, etc., etc., which I thought I needed to have to host all my own sites on.

I wanted the big daddy. I colocated it and it ended up costing me at minimum 300 a month, sometimes up to 700. I was getting jobbed.

Now, I'm paying 130 a month for all my sites.

Anyway, I have this big daddy server sitting in my dining room, collecting dust. Would it be possible for me to host four new websites from home using this server?

The problem I will obviously have is connection speed. I'm using SBC Yahoo DSL right now and I can up the bandwidth both ways for 20 bucks a month or so...

Would a visitor to that page see a page that loads at an acceptable rate? Say a 10kb page? Let's say I'm getting 800 unique visitors a day to the server who view 3 pages per person.

What else can I do with this server aside from selling it on Ebay and hoping to get back what I owe.

(I didn't know what I was getting into when I bought it 2 years ago. I was and still am basically, a newb)

jenkers




msg:961858
 6:43 pm on Apr 27, 2005 (gmt 0)

Hi Shortbus,
with that spec you could host dozens of websites. It all depends what your serving up of course. If its mainly static html then you could host loads of websites.

Your main problem is always going to be maximum concurrent users due to the fact that most residential services have relatively smaller up pipes.

But, that doesn't generally become a problem until you're getting lots of hits.

With your example of 800 visitors * 3 page views * 10k per page - theoretically you'd have no problems providing they don't all visit at exactly the same time - but...with a residential service you are always sharing with other users - at peak times you could have problems.

Namaste




msg:961859
 7:22 pm on Apr 27, 2005 (gmt 0)

get a 2nd line from another ISP and loadbalance them

FourDegreez




msg:961860
 8:01 pm on Apr 27, 2005 (gmt 0)

Think of it this way, if you've got "384Kbps" upload, that's realistically about a 40KB pipe. If your pages average 10KB, you can essentially serve four pages simultaneously (that is, within the same second). Theoretically that's 240 pages per minute, or 14,400 pages per hour. Of course in reality, peak hours aren't evenly-distributed throughout, so maybe if you start serving close to 10,000 pages per hour you'd start running into trouble.

One goal to aim for when bandwidth is your limiting factor is keeping page sizes as small as possible. =)

Namaste




msg:961861
 8:32 pm on Apr 27, 2005 (gmt 0)

and zip them

that brings up another point, home based hosting is not practical for image based sites...text based like boards is fine.

shortbus1662




msg:961862
 8:54 pm on Apr 27, 2005 (gmt 0)

This is a very helpful thread people. I really appreciate the advice given so far and look forward to reading more...

GoogleGuy




msg:961863
 5:35 am on Apr 28, 2005 (gmt 0)

Most excellent post, mack.

listerine




msg:961864
 8:30 pm on Apr 28, 2005 (gmt 0)

"... you can essentially serve four pages simultaneously ... Of course in reality, peak hours aren't evenly-distributed throughout ... "

well, thats not counting all other traffic and TCP overhead.

plus, when spider from google/yahoo/etc. hit your site, they most likely put N page from your site (that they plan to index) in a pool and give those to M spiders who will access it in parallel. this is in addition to whatever traffic human visitor/other bots brings to your site.

another thing, depend on your provider, your site might be fast for you , but EXTREAMLY slow for the rest of the world, so before you put your home data center online, make sure you ask people around from different states to test it out.

mikelly




msg:961865
 6:15 pm on Apr 29, 2005 (gmt 0)

I thought this was a newbie section...

I hosted from home in around 1995, and I did it for 5 years. Currently I am a hosting provider and I lease the servers.

Leasing is a much better deal because of the issues already mentioned such as having someone else always upgrading your software, actual redundant load balanced backbone connections, automatic backups stored offsite, lower costs, 24 hour support staffs, backup power generation, and the list goes on...

As for how many sites you can host on a server at home or away, most people use less than 50MB of space on a large site and less than 500MB (way less) of transfer. But they over estimate thier usage by a factor of about 100 unless they are streaming or storing graphics/music.

Someone in this thread mentioned 3/4 terrabite, I can't even imagine using that much storage space. That would hold everything I will see in my entire life (or about 75 years worth) at 250x250px @60 FPS

Databases can also get quite large.

The people in this forum are probably an exception to this rule.

As a summary, if you are just going to need a few dozen servers then leasing is the way to go because of the infrastucture costs.

jenkers




msg:961866
 9:03 pm on May 1, 2005 (gmt 0)

sorry if I muddied the waters - I was just trying to explain to another poster that as I needed so much diskspace it was not possible to use a dedicated server - far too expensive (I don't need 3/4 tb for storage but i run multiple iterations of link analysis against millions upon millions of URLs and the temp files for doing this get very large).

stephanwehner




msg:961867
 9:49 pm on May 12, 2005 (gmt 0)

I was a little surprised to find "Click on start > control panel " in the article. For hosting from home I use a Linux machine. If I were to start out now, I'd go with freebsd or openbsd.

Stephan

mack




msg:961868
 12:53 am on May 13, 2005 (gmt 0)

I was a little surprised to find "Click on start > control panel " in the article. For hosting from home I use a Linux machine. If I were to start out now, I'd go with freebsd or openbsd.

Agreed, thats why the start > control pannel part was explaining how to set up a windows pc to live on the same lan your servers (linux) when you require static ip assignment to all systems within your network, when not running DHCP from the router.

Mack.

This 31 message thread spans 2 pages: 31 ( [1] 2 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / New To Web Development
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved