managing thousands of html pages

[java2king]

I have 3 websites mirroring the same content, i have 73,000 html pages - of course I use SSI and css to manage them.

Also, I do not use dedicated hosting - loading the website on ftp alone takes about 25 mins when I want to change something and then uploading anything takes forever - I think I do not want to move to dedicated or virtual dedicated hosting.

Any gurus here have suggestions on how I can ease my management - I don't want to make the site dymanic (php/asp) for now.

Is there someway I can compress the size of the files.

[txbakers]

if you make the site dynamic, you can probably manage all that in a few actual web pages, with a good database storing all your content.

Much easier to maintain the database than 73,000 pages of web content.

[2by4]

You should bite the bullet and make it dynamic, there's lots of ways to do it without using databases, which if you're on a shared server might get overloaded.

Easiest is to create template sections and then fill them with the correct content/navigation using query string parameters, running behind mod_rewrite to keep all your urls the same.

I never make sites non-dynamic anymore, even small ones, it's just easier to start dynamic in some form or other than to switch large numbers of pages later to an easier to maintain format.

[mblair]

Yea, that really screams to be dynamic. With mod_rewrite your users wouldn't no the difference.

But, if you must keep it static, are you using SSI in teh optimal fashion? The reason why I ask is that if done optimally, you shouldn't have to update that many pages at one time as your 'common' code could all be in sets of includes. The only thing that would be in the 73,000 pages would be unique content that way.

There are ways to compress HTML pages to get rid of whitespace, etc -- but I don't think that the few bytes you would sacrifice doing that would justify the total lack of readability you would find.

Another option would be to compress/archive up the whole site, and uncompress/unarchive it at the destination server. That will greatly speed FTP time as it will be one file and will be well compressed. But that is an awful lot of files to uncompress... I've never done that many myself but it might work :-)

[Alioc]

I have the same problem. More than 100K html pages. I zip the files here on my computer, upload, then unzip on the server side. I zip each folder as a standalone file. One big file would be very problematic. Upload may crash, your server may not respond, etc. Also, if your server space is limited, you can stay within the limits by uploading & unzipping files in groups than deleting the zip files as you unzip.

I know it's a bit pain but I really feel much better this way because everything is under my control; while switching hosts, if the server goes down with no backup, etc.