|Setting up a home test server|
To test your projects.
Setting up a development server for testing purposes.
Most web developers will use a web hosting company to host their websites. It makes perfect sense to outsource the most critical aspect of your web presence to professionals. Web hosting is one area where you can not make do with good, it has to be great.
With that said it can sometimes be very beneficial to have a test bed where you can develop your site locally without having to actually upload your work to your web host. If your site is simply a static site written in html then there really is no need to use anything more than a web browser and place your files in a folder on your computer. But what if you want to be able to test new code and scripts before they are sent to the server. You could simply create a new directory on your web host and use that for development, but then you need to spend time uploading, testing, editing then uploading again, it can be a very time consuming process.
This is the perfect job for a test server. A test server can either be a local server installed on your PC, or a stand alone server on your LAN (local area network) My preferred method is to have a server on your network. That way it is very easy to work from any pc on your network and have direct access to your development server.
Another factor that I think is very important is to choose and configure your development server to be very similar (preferably identical) to the server that is used by your hosting company. I don't mean the hardware specification, but the software and configuration.
If you can keep your development server as closely configured to your production server then there is less chance of you having compatibility issues when you move your content to the production server.
Your test server can contain all of the features of your production server (in most cases) for example you can install Mysql, PHP and perl to let you test and debug your scripts and code. On Windows servers you can also install ASP for example to let you test your asp/asp.net coding.
The test server I have set up is a Linux box that was originally an old PC. Recycling your old computer into a development server can be a very cost effective way of achieving this without having to spend lots of money in the process.
In terms of your operating system, if you use Linux you can choose to purchase a distribution or in many cases download the distribution from your provider of choice, for example Debain, Suse and Redhat are common Linux providers.
If you want to run a Windows server then it will almost certainly cost you a little more, due to licence costs. In terms of choice, most windows hosts run Windows 2000 or Windows 2003. It is possible to use Windows XP (pro) as your server because it comes with IIS (internet information sever) as an optional extra. However the version that comes with XP pro is a little limited, you can only run one site on it. If it was being used as a web server this means only one site could be hosted. In our development environment this means you can only have one development site hosted on it.
If you run Win 2000 or 2003 you can set up multiple development sites.
It is also possible to install the Apache web server with Windows, although this may not be desirable. Not many hosting companies will offer this combination. Remember we are trying to keep our develop server as close in specification to our production server as possible.
With Linux you are not limited to how many sites you can place on the server. What I tend to do is have an account set up on the server for every site I work on. Then I can access the sites vie http or ftp etc across my network.
What will you need An old PC (or a new one of you are feeling flush!): You can easily convert your old pc to act as a LAN server for testing and development work.
A network hub/switch or a router with built in hub/switch: This is the key components that will allow your server to communicate with other computers on your network.
Network cards: most modern computers will have at least one of these. Older computers may not have one installed and it may be necessary to acquire and install one. Most NIC cards will work with Linux, but it is important to find out before you buy. Network cards became standard on most computers with the broadband explosion. Older systems may not have had them because connectivity then was generally dial up modem connection.
Network cables: You will need these to provide the connection vehicle between your different systems. Think of this as your "tiny internet". The exception to the rule is if you use wireless technology. In this case you will need a wireless router and a wireless network card for each system.
An operating system: Windows or Linux, use what is currently deployed on your production web server.
Once you have everything physically in place it is time to actually set up the server. Your first step will be to install the operating system. Either from a CD(s) Windows/Linux or online using an ISO image (Linux).
Installing an operating system can (as you know) take a little time but is generally pretty straight forward. During install windows will ask you some questions and ask for a product key. Linux will ask you for package information. At this point you may want to pick your software options such as apache, PHP, Mysql perl etc. You can however opt to wait until you have a functioning system and add your packages then.
Select the Apache web server and an FTP server. Common examples will be Vsftp (Very secure ftp) or Proftp. Both are very good FTP servers. Choose one and select it to be installed.
It is also a good idea to install the apache docs. Not only does this give you plenty of reading. It also gives you plenty of pages to serve for testing purposes.
Windows is different. You will need to add all third party software and applications after you have a functional system. Windows ships with only the operating system and a mostly MS tools. Linux distros on the other hand contain literally thousands of opensource projects.
The procedure for installing software on Linux is different to windows. Most Linux distributions will come with a package manager or control panel. The terminology tends to vary from product to product.
A package manager is a very useful tool. It enables you to install a wide range of applications and not worry about dependencies. A good example is "Yast" that comes with Suse Linux. It allows you to install software using a GUI (Graphical user interface). It allows you to search for software that is included with the distributions cd's. You can then select what packages you wish to install and it does it for you.
Most Linux companies provide similar package managers for their distributions. This makes it so much easier to manage your system. With Linux a lot of software relies on other pieces of software for it to work. This can be a real headache. Package managers take care of this by installing everything you need to run your chosen applications.
Setting up connectivity
My LAN test servers have always been Linux based. Linux is he most common operating system used by web hosting companies because of this I have went into a lot more detail with regards to setting up a Linux server. If you have an existing network then it really is as simple as plugging an extra network cable from your existing LAN into your "new" server. This will make the physical connection. You now need to configure your servers connection settings.
There are things you will need to know. Your router IP address
The IP address you wish to assign to the server
The subnet that your server will reside on.
Name server 1 address
Name server 2 address
At this stage I am assuming you have a computer system set up and running a Linux operating system. Your system already has a web server and an FTP server installed. Before we can even think about viewing anything across the local area network we need to enable the server to connect to the network.
The first thing you should do is to access your Linux system control centre. If you are using Suse Linux this tool will be called Yast or Yast2. Almost all Linux distributions will have some form of Control tool that allows you to configure your system using a user interface.
If you are unsure please have a look through the manual that came with your Linux software or consult their website.
The first configuration option that we want to access is network settings. From there we will tell the server how to become part of our network. You should choose static ip address. and give your server an IP that is logical to your existing network. For example if your router is 192.168.1.0 and your PC is 192.168.1.1 you should assign 192.168.1.2 to your server.
The layout of your network settings properties box will vary depending on what version of Linux you are using. Typically you will be asked for 2 name servers (these should be provided by your ISP) an ip address, possibly a subnet address and a gateway address.
The gateway address is the internal IP address of your router. Once you insert your correct details save your settings. In theory, and if all went well you Should now have your server running on the network and capable of serving a development site.
Go to another pc on your network and type in the ip of the server and see what happens. (remember [)...] If you see the apache test page then you have successfully set up your server if not then we need to take a step back.
The most common reason for the page not showing is apache is not running. open your shell (console) and type
It should talk back to you by saying "httpd started". If it does this then return to your other computer and try to access your server again. If it works then this time you are in luck. If not then you need to check a few things.
The obvious thing to check is that your network cables are all correctly attached. You should then double check your network settings in the Linux control area. Because there are so many different variations of Linux is is impossible to cover them all in any great detail. I am only able to talk in very general terms. Please consult any documentations that came with your Linux distribution. There may also be help available from within your control area.
If you are now able to view your test server from other computers on your network then you are ready to proceed to the next stages
It is not very often (if ever) that I will recommend a tool or app but in this case I will break with tradition and recommend a free server application called Webmin.
Webmin is a server control panel that is designed for server administrators. it is not a control panel like Cpanel that was developed for the hosting industry. It is designed to enable you to configure your system, start stop services and edit your configuration files using a web browser. Once you get to know this tool it will really help you and and speed up system administration. I will not go into any detail about installing Webmin because the documentation and instructions provided are excellent. www.webmin.com
What you will notice when you access your Webmin install is that it allows you to start and stop almost any service without having to physically go to the server, or shell into the server. You simple click on servers and all your options will be displayed. Lets click on proftp ftp server. At the very bottom of the page you will see a button "start proftpd" click this button and your ftp server will start.
How lets imaging you are working on a website called "Car site" what I would tend to do now is create a user account on the server called "car". Again this is done through your Linux control tool. You will be asked for a user name password and perhaps some additional information about the new user. Once the account has been created you will have a fully functional Linux user account called "car"
You should then set up a virtual host for your account Placing the following on your httpd.conf will set up a name based virtual host.
ScriptAlias /cgi-bin/ /home/car/public_html/cgi-bin/
<VirtualHost *> --- The first line is simple the opening tag for the virtual host.
ServerName car --- This specifies the name of the user account. It can in theory be anything but it makes since to use the same name as the user account.
DocumentRoot /home/car/public_html/ --- specifies where about on the system the users web documents will be stored.
ScriptAlias /cgi-bin/ /home/car/public_html/cgi-bin/ specifies the locating of the users cgi-bin. This entry will not actually create the cgi-bin you will still need to create the actual folder (public_html/cgi-bin) either using Shell or an ftp client.
</VirtualHost> --- the end tag of the virtual host.
You can set this up entirely through the Webmin control panel but this is one area where I feel it is probably quicker to simply use shell.
Always make a backup of your httpd.conf file before you make any changes.
You will then need to restart apache for the changes to take effect. You can do this through Webmin or by running the following in shell as user root...
In order to access this users web space account you can visit server.ip.address/~car.
On Linux system level the same location will be hosed at /home/car/public_html You should be able to ftp into your server and log in using the username "car" and the password you choose when you created the user account. You should now be able to upload files to the web folder.
I always think it is a good idea to have a different user account for every development site you work on. This just helps keep everything separate and more importantly it makes it very unlikely you will ftp content into the wrong location if your username is the same as the site name you are working on. I think it is a very bad idea to FTP as user root. The ability to FTP as root is disabled by default, if you do wish to do this you will need to alter your ftp configuration.
If you wish to take advantage of server side scripting such as PHP or perl you will need to install the languages onto your server. If you want to install it from your Linux distribution you can simply open your Linux control panel and open up your "add software" section. From there you can select PHP and Perl and let your Linux distribution update it's self by installing the chosen features.
The other way of installing such packages is to download the latest versions from their related sites and install then manually. Be warned this is a certainly not a task for someone who doesn't know what they are doing. Be sure to read the documentation provided. Linux is certainly not the easiest operating system to work with, so why make it any harder than you need to.
If your are using a router make double sure that your ports are locked down. Make certain that port 80 is closed. If you left port 80 open it is possible for web traffic to enter your development server. This is almost certainly not desired.
You can shorten the URL you use to access your project to simply [car...] by placing the following in your hosts file.
On the web servers are required to run 24/7/365. Your development server is only required when you wish to view your development work. You can simply turn it off when not required.
Having a development server can be of great assistance. For example imaging you have a template driven site and you want to test a new layout. You have 30 or so files that need uploaded to your web server on the other side of the country. maybe a few mins to upload it. On your LAN server this would only take a few seconds. Over a network everything is so much faster. It will also help you to appreciate the hosting that we all take for granted.
An excellent post!
|This is the perfect job for a test server. A test server can either be a local server installed on your PC, or a stand alone server on your LAN (local area network) My preferred method is to have a server on your network. That way it is very easy to work from any pc on your network and have direct access to your development server. |
I'd say for most cases, e.g. for the majority of classical "LAMP"-style applications, there's no need to go to the trouble of setting up an extra physical server. A local Apache / MySQL etc. installation on one's main PC should do the job just as well, and it should be no problem to set up an environment basically identical to the webserver host (same server versions, software packages), which would also be accessible from other points on the LAN.
I develop several large, data-intensive sites on my workstation (a Linux machine), or even on my laptop (OS X) when on the move and have never had the need for a seperate test server. I haven't tried this kind of development on Windows though, so can't say if it would be practical there though.
Of course setting up a dedicated test web server is excellent practice for when it comes to dealing with the real web server.
|Make certain that port 80 is closed. If you left port 80 open it is possible for web traffic to enter your development server. This is almost certainly not desired. |
Another thought: if you're say a web developer with a home office on DSL or some other non-fixed internet connection, and need to demonstrate your development environment to others (partners, customers) you can use one of the "dynamic DNS" services to get a "real" hostname, and point this to your development server. Of course you'll have to be careful what traffic you let through on Port 80 (some kind of password protection is a must), but it's a good way of getting a test environment online without messing about with the production webserver.
Outstanding post mack! I've flagged it already.
Another possibility I've been playing with recently is to use virtual PC software to test out operating systems and servers. If you've got a good workstation and a lot of RAM you can make a virtual machine on your workstation and play with it to your heart's content.
You mentioned the cost prohibitive Windows environment. If you can afford a subscription to the MSDN (Microsoft Developer's Network) you can get access to several different servers and operating systems for testing. In my case it was less than a new low-end PC. They even include virtual PC software in the package, although I already had VMware.
The great benefit of a virtual environment is that you aren't going to destroy a real PC if something goes wrong. I can create snapshots of the OS and return to that at any time. If I mess something up I can go right back to a previous snapshot. This is great for trial-and-error learning on almost any OS (no MAC).
i started thinking about making a setup like this a few days ago, so this is a very timely post
im decided on installing Ubuntu and php mysql and perl on a 70$ computer from ebay (P2 400Mhz, 250MB Ram). According to Ubuntu forums this is sufficient.
my main concern, which wasnt covered here but is probably beyond the scope, is to integrate this testing server with 3 other machines in a WIFI environment
I think you answered my question by saying "a router with a built in hub/switch" compatible with Ubuntu is needed - thanks!
|However the version that comes with XP pro is a little limited, you can only run one site on it. If it was being used as a web server this means only one site could be hosted. In our development environment this means you can only have one development site hosted on it. |
You can work around this by setting up each website as a seperate folder under inetpub. When you want to access them locally it's like this
remove the space it's there to keep it from being a link
I was going to joke around and ask "Where do I send the check?" (for the timely consultation) but, instead, let me simply say: Thank-you.
Thank-you to everyone who labors and gives their time so generously in order to share their knowledge and experience.
|If you left port 80 open it is possible for web traffic to enter your development server. |
If the test machine is on a LAN with "residential" internet access, it's possible (I'd say even probable) that the ISP blocks inbound Port 80 requests, to make it (a little) tricky to host a web site. But if you're looking to keep your development website private, that's just what you needed.
I'd just like to add a comment.
I find that having a low powered web server is actually preferable to having a faster one.
On your live server, when multiple clients are connecting at the same time, the stress of your web application is multiplied.
On your development server, you won't see this as you'll likely be the only person using it. Therefore any performance bottlenecks could go unnoticed. I personally use a Pentium II, 300Mhz as my development server and that's a bit too powerful.
I also recommend a staging (testing) server as well.
Your development server should be entirely for your own use. This is your construction site where things go on that might scare your clients (broken links, dummy text, big error messages, etc.)
Once you have a stable build, transfer it to the staging server and give that address to the client whilst you continue to work on the site on your development server.
Hello, what an interesting subject!
I'm using an old PII laptop with Linux and W2K (327Mb RAM) to debug some ASP website.
I wonder if you would have any tips & tricks regarding IIS 5.0. Should I fight to get IIS 6.0? But it's another subject...
Let's not forget XAMP (Apachefriends.org) and WAMP (http://www.wampserver.com/en/index.php) as easy ways to set up Apache/PHP/MySQL et al. on a local Windows PC.
XAMP installs everything under a folder and outside of the Windows registry, so it is easily managed.
One must be aware however that Windows versions of some software are not equivalent to Linux versions (MySQL for example). This can lead to problems.
Additionally, with *any* development server, your test environment is NEVER equivalent to a hosted environment. When you try to move your work to a hosted server, you will encounter many issues so leave yourself plenty of time.
I currently have the following -
- DSL home connection 2mb/256
- Wireless/Cable router with firewall
- Dynamic IP address and using Dynamic dns provider so I can have a free sub-domain pointing to my network.
- Laptop running Apache, Mysql, Php5, Zend dev tool etc
- Windows box running IIS ready for future ASP learning in 2006 :)
- Lynx was on the same machine in dual boot but will remove and set up a seperate box for all apache/php dev work
- Using windows box for filing and remote access for filing.
- Port 80 used for Apache, Port 8080 used for windows IIS
- FTP currently closed until I need it for future Lynx
- Apart from firewall in router running an additional one on the windows box. Access is also password protected.
The only advice I can give is to think and map everything you intend to do before you start.
- Strong Coffee
- List all the things you want to achieve first
- Decide on the best hardware/Software for each requirement
- Make a plan of all the computers you have
- Set up simple naming conventions for each machine or service
- Have a clear table of all IP addresses for each machine
- Keep it simple and concentrate on one thing at a time.
- learn SSH for Lynx so you can add new virthosts quickly
- Don't get angry if something won't work, walk away and try again later, keep it fun
- Pat yourself on the back when done, its a job well done and to be proud of!
Time is the only problem, but it should save you money
You may be trying something new for 10 minutes and end up spending hours thats learning :)
Wow, what an excellent first post! I just happen to have an old PC spare and was wondering what to do with it. I hate developing live on my current webhost because FTP always times out. Also the server can go down and stop you working, as just happened to me for 2 days.
I have some questions though.
1. What is the best version of Linux to go for? (Forgive me, I know next to nothing about Linux.)
2. Could you use the server later on as a real server, connected to the net? Kind of like a place for extra files that only a few users might need. Or even a full site.
3. If you just plug it in to your LAN (as you suggest) can someone on the internet access it at all? (Ie: hackers.)
4. Do you need to use a network or can you have a system where your Windows and Linux boxes are simply connected by a USB cable, so you can transfer files? (I believe Windows to Windows can do this.)
5. What RAM and hard drive size will be suitable?
Lastly, in the post, there are some confusing words - can a moderator edit the post? Words like "imaging" instead of "imagine" happens a couple of times. Example:
"How lets imaging you are working on a website".
I didn't know what was meant there at first.
Sorry to nitpick on such a great post!
Also, if you only have one computer, you won't need a hub. Just plug a second network card in to your machine and one in to the development server and use a crossover cable between them. Just set up a secondary private network for the dev server.
This way your development server won't be directly connected to the Internet. You don't then have the hassle of keeping the development server patched with security updates.
-To get around the IIS limitation of one website only on Windows XP, simply install Apache.
-I second the XAMP recommendation for easily installing Apache/MYSQL on Windows.
-Depending on your hosting setup, your local server may be able to connect to the remote MySQL database. So basically you could test development code on live data. (I suggest read-only access unless you are careful).
-If you want to copy your remote database locally for testing, PHPMyAdmin allows MySQL data import/export. If you want a super easy way to sync your databases look for an application called Navicat.
-You don't necessarily need a monitor for your local testing server; just run the server 'headless'. If you are using XP Pro use 'remote desktop' to connect. Other OS's can try various free VNC software.
-If you have a laptop and travel a lot, then installing Apache/MySQL on it (instead of a separate computer on your network) is a good idea. Then you can develop on the road, even without any internet connection.
-The easiest way (IMHO) is to just develop on a Mac which comes with Apache pre-installed; MySQL is a very simple installation on OS/X. This is a greaat setup if your remote server is runing Linux because OS/X is also Linux based. You may still want to keep on old PC available on your network for browser compatibility testing. Microsoft's remote desktop client is available for OS/X and it works wonderfully for this purpose.
<added: Kudos to the original poster on very thorough post! >
a point for those not needing/using a network, the whole set-up (appache/ php/ MySQL/ phpmy admin etc.) can be installed on any pc - usually free. A quick google will turn up complete bundles downloadable free of charge. A server is not necessarily new/ recycled hardware at all. It can just be extra software on your pc
I run these on my pc for developing and building php/ database driven sites simply accessing via 'localhost'.
A simpler less demanding method I use to accomplish the same thing is a second "staging" account on a remote server. If my account is example.com then I set up staging.example.com to test things.
It's 100% exactly the same as my main account except the domain name.
The best part is, when I'm ready to push the site live I can just SSH up to the server and swap the subdirectories and it's live - if something breaks I simply swap them back.
It's not like I don't have the hardware either as I have a small armada of networked PCs laying about but setting up more servers is just more headaches to contend with instead of getting down the the business of building web sites.
No fuss, no muss, no cables, cards, software, updates, firewalls, nada.
However, it never hurts to have a fully functional local copy of your site either.
One nice side-benefit of installing apache/php/mysql on my home machine is the capability to use php to read the html code from other sites' pages. Many hosting outfits block this, but being able to do it from my home machine can be very useful. It makes it easy to check incoming links to my site for example, to confirm that people are living up to their link swap agreement with me. Php makes grabbing a page so easy with the file function:
$array_of_lines = file("http://www.somesite.com/page.html");
And of course, it's great for testing, as already pointed out on this thread.
If you try and print out the result, you get the page as if it were viewed in the browser, although with array numbers printed on top. Here's how to just show the contents of the array.
$array_of_lines = file("the site address here");
Though you might as well just view the source! (Use a decent text editor like Notepad2 to get line numbers.)
Although setting up "another PC" is the tried and true method, there is a newer technology that makes settings up test servers much easier and more flexible.
Virtualization technology allows you to create complete "virtual computers" on your existing real computer.
Each virtual PC runs it's own operating system and acts and behaves just like another computer - complete with virtual network card, virtual hard disk, etc.
The two leading programs for doing this are VMWare and Microsoft Virtual PC.
You can run ANY operating system inside a virtual machine - for example, on my system at home running Windows XP Pro and Microsoft Virtual PC, I have virtual machines running DOS, Windows 3.1, Windows XP, Windows 2003, and Linux.
Yes - you can run Linux virtual machine on a Windows PC (or with VMWare you can a windows virtual PC on physical Linux system).
The beauty of virtual PC is you start and stop the VM as often as you like saving a snapshot to disk. Sort of like a laptop going to sleep - you startup the VM days, or weeks later and pick up right where you left off.
Another use for virtualization is for testing. you can create VM's with different operating systems and web browers.
So if you need to see how your html renders on a specific version of Windows and Web Browser, you can do that.
For regression testing, you can preserve older, but still used configurations such as Windows 98, Windows 3.11, etc. and older browser versions and test your webpages against them.
Because you start/stop virtual machines, the only "overhead" to keeping all this configurations is the disk space for the "virtual hard disk files".
So instead of having a room full of PC's, each with a different configuration for testing, you just keep lots of data files (of VM's) on your system and turn them on only when needed.
Note that this works for any os or web server - Windows, LAMP, Solaris, etc. -- the technique is OS and web server agnostic.
Wow, thats backwords compatibility testing to the extreme!
For windows, apache, php, mysql development, I really like Uniform Server - [uniformserver.com...] I've used other all-in-ones, and Uniform Server is by far the best.
No installation/configuration program, just copy all the files. For developing multiple websites, just need to add a virtual host to htconf and let windows HOSTS file know about it. My development and live server (a linux machine) code just differ in the path to the php include directory.
This can go well with VirtualHost config tip [webmasterworld.com].
VMWare is a great tool and I have used it for several years. I do not recommend it in place of a local apache/php/mysql install, which in my experience is easier, more stable, and more usable.
I would not put a dev environment on Windows XP due to numerous compatibility issues (even with XAMP). There are slashes vs. backslashes, .htaccess file naming issues, and more. Use a linux box as suggested or use Linux for everything.
nm got it
Can a Linux test server be used without a screen?
You can log in via SSH (with Putty etc.) and work on the command line; or possibly use some web-based software like Webmin or PHPMyAdmin for administration.
OK, what I want to do is this: set up the server with a screen, then leave it for use without one. (I can plug the screen back in if needed. In fact I plan to share a TFT, so clearly I can't use both sockets at once. One is digital (my main PC) the other will have to be analogue.)
Once the server is set up, it'll not need to be changed (hopefully). I'll just be networking to it via a cable.
Hope that makes sense.
Thats exactly what I did when I set up my server. I used a screen to carry out the initual set up and configuration then removed the screen and simply controled the server with shell or webmin.
If I needed to I could simply plug the screen back in. If you are using a Windows server then remote desktop software might be a solution.
I use the server virtualization method and it works out great. I have a Red Hat Linux server running Apache and a Windows 2003 Server running IIS all running on my laptop (host OS is Windows XP) at the same time (it's a pretty good laptop!) Make sure you have enough memory too, at least a Gig, especially if you're like me and are going to run more than one virtual server at a time.
I have tried both Microsoft's Virtual PC and VMWare Workstation and I would recommend VMWare. Virtual PC only works with Windows OSes and I had problems with it in conjunction with wireless NICs in the past.