Managing dev,staging and live servers?

[optik]

Hi

I'm having to create a site that requires a development server, staging server and live server.

Does anyone have any recommendations on how keep everything in sync and if possible be able to upload selected file to all three or two of the servers at once.

Mac solutions only please.

Thanks

[phranque]

use a revision control system/source code management such as cvs, subversion or git.
divide your project into "modules" that can be checked in & out separately as required for the application.
design your database and directory structure such that application source code, content and data can be manage separately from user-generated content and data.

also, don't forget to exclude the search engines from your staging and development sites by requiring authentication for access or some other suitable method.

[optik]

ok, all this cvs is completely new to me, there seems to be a lot to it.

So for a start I would have an svn repository for each environment which can be accessed by multiple people?

I'm still not clear how the server updates the web folder with changes to the svn, probably through an apache module command line entry?

I'm also not clear if it would be possible to make a file change on the dev server, have that approved then with the click of one button have that change be made live on all environments and repositories.

cvs/svn seems to be more geared towards rolling out distinct version rather than synchronising small changes or have I got that wrong?

[Status_203]

What is the point in having separate environments if you deploy changes to them at the same time?

Surely you do your development and maybe code testing on your dev server.

THEN test your rollout process in deploying to the staging server (and code testing if it isn't being done in dev environment... and maybe even tested again by somebody else even if it is)

THEN rollout to live when you're happy that neither your new code nor your deployment process is going to screw things up (and even then you want that process to include a quick way to back things out if things go wrong anyway)

[Status_203]

That said... :)

I'll talk about Subversion as that's what I use.

You would want a single repository accessible from wherever you need to check out the code. It's worth learning about the splitting of a repository into trunk, branches and tags but depending on the complexity of your development it may not be necessary to use all 3.

The trunk holds the canonical history of changes. With a new project, especially one that might take some time you copy the trunk to a branch. Changes made to the branch do not affect the trunk (or other branches that other people are working on). If the project goes away discard the branch. If the project is completed merge the changes back into the trunk (Subversion helps with this. It isn't a purely manual process). When you want a release copy a snapshot of trunk into a tag (basically means you can always easily get back any version of code that you've released and easily refer to which code is to be deployed by referring to the tag name).

For deployment you have two options. Have the repository accessible to the environments, install an svn client on the servers and checkout the tag straight from the repository (I'm not doing this yet so I can't help with the details). Or hold a copy of the code locally and transfer it using sftp. I create a new folder in which to upload the code and when it is complete switch over the webroot folder and the new folder so if it goes wrong I can switch them back again. You also need to consider database backups when deploying.

As a general rule, the more you can automate of this process the more reliable it will be.

[optik]

I understand the need for testing in the different environments for large changes to the code,
and I will look into SVN/CVS for this task, but often I am required to change a line of text or an image or something else very small which doesn't need testing in every environment.

So am I right in thinking I have to install an SVN client on each server, set it up with the right repository and then use the command line to action an update for each one at a time?

[Status_203]

If you're looking to automatically deploy from the repository then yes you want a client on each machine connecting to a central repository.

The first deployment you'll do a 'checkout' of the relevant tag (or specified trunk revision if you don't want the overhead of tags) which will grab *everything*. On subsequent deployments you'll 'switch' to the new tag (or 'update' to a new trunk revision) and Subversion will drag back only the files that have changed.

That's for code changes only of course. If you're making db changes as well you'll need to adapt the process to accomodate them.

*And still don't forget your backup process before deployment. Preferably automated.*

[Status_203]

Oh yes, and just to add that, deploying changes that I thought were so simple they didn't need proper testing has tripped me up a number of times. Sometimes the risk has to be taken for business reasons, but don't skip testing just because it doesn't seem worth it.

[optik]

I hear what your saying, but say for changing a typo in some copy there isn't much that's going to go wrong. So will the svn not manage images and media files?

If the update to a trunk is a simple as a command line entry on each server that's seems quick enough anyway.

One last thing before I jump in the deep end, how would this scenario play out with svn...
say a group of people are working on the sites shopping basket, it turns out half way there is a critical bug in the old basket that needs fixing immediately. How would the bug be fixed without uploading the half done new version of the basket?

[Status_203]

Binary files can be stored in svn without problems. If they are changing in every release then storage is going to less efficient than for textual files but that's the only, trivial, issue I can think of. Svn does a cheap copy wherever possible (i.e. it doesn't create another copy of the file, it just says this version/branch/tag uses the same file as over there, until the file is changed).

Creating a new tag for every release is a little more work than just doing all your work in the trunk. However, it is then trivial to get back the version of source that has been released. Simple version - Check out the current live tag to a new working folder. Fix the bug. Commit the changes and rollout. Merge the changes into the version you are working on.

(Note that if making "full" use of svn and branches for changes etc you might want to keep tags inviolate and never commit to them, instead going from branch to trunk to new tag every time, however that does create a lot more complexity and overhead if your release priorities ever get switched around)

[optik]

Ok.. thanks for your help, I'll get this setup and have a look in more detail.

[optik]

I've installed subversion on my server and set up users and a repo folder but can't seem to access it from my client.

I'm using svn://[user]@[host]/svn

For user i'm using the SVN user I created in the conf file, for host the server IP and the repo folder is "svn" on the root level of the server.

The connection times out using svnx

[optik]

I've connected now with ssh ok

[optik]

Ok so I now have a repo setup, a working directory, and seem to be able to commit changes.

But how do I get the updated repo to be seen on the live site?

I can see that you can point apache to the repo folder, but what about the different versions or are they saved as separate repos?

[Status_203]

"Checkout" to the web root the version you want live (The folder from which Apache serves up pages).

Make sure Apache is configured not to serve up files and directories starting with a "." (It usually is by default). This is because Subversion working folders contain a ".svn" folder with information about the version checked out to the working folder.

[optik]

How does the dev site on a different server read the repo?

[optik]

I'm not sure how I checkout just the trunk to the web folder either?

[Status_203]

If they are on the same network then (...configure firewal and...) point at an address that identifies the machine as well as the path.

If they're not then you need to sort out some sort of server. I believe Apache is common but I haven't done this myself. I'd start here [subversion.apache.org].

As for your other query just google svn checkout trunk. There's plenty of resources out there.