Other Topics

© Copyright Robert Vasvari, 1993-2010.

Warning: always avoid using non-ascii characters (like å or ü and others) in filenames. As you run the sync between different hosts, these names will cause problems. Again, only the names of the files should be restricted, the contents can be anything.

Folder Syncronization

Who is Folder Sync For? Folder sync has two primary uses:
  • Website maintenance - Normally you have on your desktop a copy of your website that you edit. Whenever you think you done with your changes for now, you want to publish to your website. This is extremely easy with RBrowser, sync RBrowser's sync engine will automatically find your changes and publish your new and changed files only to the remote server. You may sync as often as you like.
  • Backups - Make copies of your important data to remote servers or to others disks on the desktop system. Either way it is the same easy-to-use interface!

Folder Sync means a very simple thing: You have a source folder whose contents you edit. Whenever the sync operation is performed, the target folder will get the deltas only, so changed files are copied to the target folder. This sounds very simple, but first we have to clear the air about the different kinds of Folder Syncronizations that are possible. Here is a quick explanation of terms:

Source Folder: The folder whose contents you change (like your local copy of your website). When you are done changing it, you run a Local-to-Remote sync to publish your changes.

Destination (Target) Folder: The folder that you do not touch directly, the sync engine will publish your changes from the Source Folder into the Destination Folder. This target folder can reside on your local machine or on a remote host on which you have an FTP or SSH/SFTP account.

Phase 1, Sync by Source Database: In this phase, the target folder is not looked at at all, the source folder's state is compared to the source folder's previous state. The new state is then stored in a Database file in the source folder under the name .rbtp_sync_db. This method has various advantages: the speed first of all, since there is no need for network access. Secondly the accuracy: we do not have to put up with the differences between various systems, comparing file names checksums, mod dates are relatively messy across the internet. The source Database comparison is very accurate, it can easily compare checksums and resource fork data too. The remote target folder is then accessed just to upload the changed files. The question is, is this enough? In most cases, yes, if we are reasonably certain that the target folder is there, it is more less in sync, and it is not being altered by somebody else!

Source-Destination Comparison (Turned on by the Always Check Destination checkbox): The target folder is accessed by some protocol, like FTP/SFTP/SSH, then source and target files are actually compared by the most accurate means possible. FTP is a poor protocol, but SSH actually allows a very accurate comparison, since checksums of the remote files are available on most systems. This phase is more time-consuming then Phase 1, but, when it completes, one can be reasonably sure that source and target are in sync.

More on Sync Options...

In this case both source and target folders reside on the local workstation. Either one can be on a mounted external drive, the point is that there is NO network access involved in reaching either source or target folder. Many other sync apps call the target folder "remote folder" mistakenly even though it actually just another locally mounted folder.

In this case the target folder resides on a remote host. There is no need to have any custom software run on the remote host, all you need is a simple FTP or SSH/SFTP account. When you specify the target folder, the URL will contain the remote host, protocol (FTP or SFTP), username and full path of the target folder.

In this case the source folder resides on a remote host. This kind of sync operation requires SSH/SFTP account. FTP cannot support Remote-to-Local(RTL) sync, because the protocol is too poor to provide the necessary information. When you specify the source folder, the URL will contain the remote host, protocol (SFTP only), username and full path of the source folder.

One-Way versus Two-Way sync
By design, a sync definition represents a one way sync. That means, there is a source or (master) folder where you do your work. This folder is active, it changes. Then, on the receiving end of the sync, there is a passive destination folder that (hopefully) does not change, it simply receives changes from the source folder. So, how about when both source and destination get edited? In other words there is no "master" folder and we want the changed files in both folders to get synced into the other, much like phones do?

This is one of the most recurring questions. One must realize that a computer folder is not a telephone directory where contact records can be synced both ways. In a computer where files have naming conflicts and many other problems a two-way sync is not really feasable, it is simply to dangerous.
There is nothing to stop the user right now to set up two sync defs that are opposites of each other, that in fact would be a two way sync, but it is would not recommended at all! A two way sync is the fastest way to accidentally lose the files you are working on.
What is the solution then? It is to reorganize your work a bit... make sure you do not work on the same file on both locations, and if so, then use source code control like cvs or something similar. Name your folders/files differently in each location then you lessen the chance of files overwriting each other. The best way to deal with a two way sync is simply to avoid it altogether.

How does the synchronization process work?

RBrowser automagically finds the changed files using various methods (Phase 1 and 2 described above), then recursively goes through the source folder and transfers all the changed files. Files are always transferred whole.

FTP or SFTP-SSH can be used in remote sync operations.

Here is the Sync Panel:

The first task is to define a Source->Destination(Target) folder pair. This is super easy, just drag a folder from the Finder or from any RBrowser viewer into the filewell and drop. One of the folders must be local, as Remote-to-Remote sync is not supported. Set various options, then click "Save" to save the folder pair. At this point you are ready to run a sync operation. The bottom table shows all sync pairs that have been defined already. The Mode column shows the abbreviation of the sync mode (Auto, P1, P2, or 'M' for Mirror), the Type column shows the type of sync operation: L->L for Local-to-Local, L->R for Local-To-Remote and R->L for Remote-To-Local(RTL).

Things to know about Remote-To-Local Folder Sync

There are a number of limitations to RTL sync: FTP as a protocol is simply too limited to support RTL sync. This is due to the fact that most FTP servers reset modification dates when a file is transferred to it, checksum is not available and size alone simply cannot detect change to a file. Therefore, RTL sync is only available in using the SFTP protocol. RTL sync is a simple one phase operation, both remote source and local target are scanned once. The other limitation is that if a file is simply renamed in the source, RTL sync still has to treat it like a new file, so it is downloaded as if it were new. This is due to the fact that a rename cannot be detected deterministically just by reading remote data, since you can have more files with identical contents.

Things to know about Local-to-Remote Folder Sync

Symbolic Link handling:

Symbolic Links are special kind of animals. They are not files, simply a pointers to a files or folders. The sync operation creates links identical to the ones in the source folder, the target of the link does not get copied! In one case, Local-to-Remote sync through FTP, this is not possible because FTP does not include the capability to create links. In this case links are ignored.

Case sensitivity Be sure to understand whether the source and the target folders are on case sensitive file systems. If your source is on UFS and have files like foo.html and Foo.html, and your target is an hfs file system, these files WILL overwrite each other.

Here is a summary of Folder Sync features:

  • Local-to-Local(LTL), Local-To-Remote(LTR) and Remote-To-Local(RTL) Folder Sync
  • Mirror or Archive mode (the default)
  • Scheduled Backups
  • Any protocol (SFTP or FTP) can be used
  • Changed filenames are detected so unnecessary file transfers can be avoided.
  • Easy drag-and_drop definitions of sync pairs.
  • Any number of simultaneous sync operations!