homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / WebmasterWorld / New To Web Development
Forum Library, Charter, Moderators: brotherhood of lan & mack

New To Web Development Forum

How to sync?

5+ Year Member

Msg#: 4176388 posted 3:18 am on Jul 26, 2010 (gmt 0)

For years, I've maintained my rather inactive web site solely by manual FTP. It's gotten large enough that I'm finally trying to use a synchronization tool. OK, it got that way a long time ago and I'm trying to catch up.

Since I've been using WS-FTP, I fired up its sync tool. What I found was that it thought nothing on the site matched my local files. Turns out this is a combination of things the FTP server on my web host doesn't support. It doesn't support MDTM, so the server file timestamps are the upload time. It doesn't support any message digest such as XMD5 or XCRC, so the client cannot match files using that.

I tested a second web host that I use for another web site, and it has the same limitations.

So how do FTP-based synchronizers work? I've spent a couple of hours searching the web, for nada. Many sync programs exist, but when you try to find out how and whether they get around these problems, they just sort of wave their arms around.

So is this situation normal? Are sync programs able to handle it? If so, how?

It's easy to see that with a message digest plugin on the server, matching is trivial. I can also see why webmasters don't want this overhead.

It's easy to see that, even with a one-minute granularity, one could use timestamps to determine whether files match, albeit with a small chance of failing to catch a mismatch.

And I can see that a more sophisticated program could keep a database of the timestamps on the FTP server: after initially uploading a file, request info on the file, enter into a database the file ID and the timestamps for the local and remote versions. Then on sync, check the local and remote timestamps and see if they match based on the DB record that links them.

The last of course would require the entire web site be re-uploaded to prime the database. I'm happy to bite that bullet if it's the right one. Whether it's the right bullet to bite is what I'm having trouble determining. The docs for WS-FTP are minimal and don't tell me. I've looked at the web sites for a number of other sync tools and none of them tell me.




WebmasterWorld Administrator mack us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Msg#: 4176388 posted 2:24 pm on Jul 30, 2010 (gmt 0)

I feel your pain on this one, its can be very problematic to find a workign solution, because there are only so many tests you can actually do before trying it live, then you really don't want to know things haven't went to plan. I recommend you do a full backup of the live site (as it is) before trying to do anything, save it in a different location to your development files. (the ones you want to sync) then if things do go wrong all you need to do is recover your downloaded backup.

Is your site, mainly dynamic or static. You mentioned the database, perhaps mysql replication is something that might be of help for this? it wont back up your actual files, but will keep an up to date copy of the database. and sync that.



5+ Year Member

Msg#: 4176388 posted 4:01 pm on Jul 30, 2010 (gmt 0)

Thanks. It's all static. When I mentioned database, I was speculating on how FTP sync might work. My master copy is the local one, so if I totally screw up the online one, I just wipe it and re-upload. But that takes a while.

I posted a similar question for WS-FTP and have had no response. I think I'll investigate some of the programs which specifically advertise synchronization as their main feature, as opposed to an afterthought as with WS-FTP. Googling <<FTP synchronization>> gets lots of hits.



5+ Year Member

Msg#: 4176388 posted 3:19 pm on Aug 4, 2010 (gmt 0)

Maybe check WinSCP, it has a checkbox for each transfer to "Only upload new or changed files". Not exactly what you are looking for, but it might help. Very handy tool.

Global Options:
 top home search open messages active posts  

Home / Forums Index / WebmasterWorld / New To Web Development
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved