Scp Sync Folders

Documentation » Using WinSCP » Basic Tasks » Synchronization (general) »
Download

This article contains detailed description of synchronizing files. You may want to see simplified guide to the process instead.

Scp will replace all existing files; hostname will be hostname or IP address; if custom port is needed (besides port 22) use -P portnumber. (dot) - it means current working directory, So download/copy from server and paste here only. From man scp:-r Recursively copy entire directories. Note that scp follows symbolic links encountered in the tree traversal. So if you have sub-directories inside localdir, the last example will only transfer files, but if you set the -r option, it will transfer files and directories.

Scp all files and folders

WinSCP can synchronize content of local directory with remote one or vice versa or even mutually. Find the function in Commands > Synchronize. Synchronize dialog will appear, where you can select synchronization mode and configure options.

Once you confirm your options, WinSCP will collect list of differences in between the directories. If option Preview changes is enabled, you will be presented with the synchronization checklist, where you can select those synchronization actions you want to actually apply.

You can select files and directories for synchronization the same way you select them for file transfer.

Before using the function make sure that WinSCP knows correct timestamps of remote files, to avoid losing data.

If you use WinSCP to synchronize your files only, you can use /synchronize to quickly open Synchronize dialog.

There are three possible directions (targets) of synchronization.

With Local direction, changes from remote directory (source) are applied to local directory (target). Only the local directory is modified.

Remote mode is the same as Local, just in an opposite direction: changes from the local directory (source) are applied to the remote directory (target). Only the remote directory is modified.

In Both mode, both local and remote directories can be modified (both can act as both source and target).

Scp Sync Folders

There are three modes of synchronization.

With Synchronize files mode, files that are newer in a source directory than in a target directory are transferred to the target directory. Files in the source directory not present in the target directory are by default transferred as well (disable this using the option Existing files only). Files in the target directory not present in the source directory can optionally be deleted (enable option Delete files). In Both mode (see above), files not present in an opposite directory are considered new, hence they are transferred (unless Existing files only is enabled), but never deleted. In other words, in Both mode, no file is ever deleted.

Advertisement

With Mirror files mode, different (both newer and older) files in the source directory are transferred to the target directory. Otherwise the mode is the same as Synchronize files.

With Synchronize timestamps mode, timestamps of target files are updated to match timestamps of source files. It will not do any transfers, nor delete anything. Simply, whenever it finds the same file in both directories, it updates a timestamp of a target file to match the one of a source file. In Both mode, it always updates the older timestamp. The mode is available with SFTP protocol only.

To automate synchronization use scripting command synchronize or .NET assembly method Session.SynchronizeDirectories.

Last updated on August 20, 2020 by Dan Nanni

Question: I want to download (or upload) files from (or to) a remote server using the scp command. In this case, I want to skip existing files, so that they will not get overwritten by scp. But the scp command would blindly overwrite existing files if the same name files exist at either host. How can I copy files over without overwriting existing files, so that only new files are downloaded (or uploaded) by scp?
Scp Sync Folders

Suppose you have a list of files on a remote host, some of which already exist locally. What you want is to transfer only those files that are not found locally. If you blindly run scp with wildcard, it would fetch all remote files (existing as well as non-existing files), and overwrite existing local files. You want to avoid this.

Sync

In another similar situation, you may want to upload local files to a remote site, but without replacing any remote files.

Here are a few ways to skip existing files when transferring files with scp.

Scp Sync Folders Mod

Method One: rsync

If the local and remote hosts have rsync installed, using rsync will be the easiest way to copy only new files over, since rsync is designed for incremental/differential backups.

In this case, you need to explicitly tell rsync to skip any existing files during sync. Otherwise, rsync will try to use file modification time to sync two hosts, which is not what you want.

To download all remote files (over SSH) while skipping existing local files:

Similarly, to upload all local files (over SSH) without overwriting any duplicate remote files:

Method Two: getfacl/setfacl

Another way to scp only new files over to a destination is by leveraging file permissions. More specifically, what you can do is to make all destination files 'read-only' before scp transfer. This will prevent any existing destination files from being overwritten by scp. After scp transfer is completed, restore the file permissions to the original state. The ACL command-line tools (getfacl and setfacl) come in handy when you temporarily change file permissions and restore them.

Scp Sync Two Folders

Here is how to scp files without replacing existing files using ACL tools.

Scp Sync Folder

To download all remote files (over SSH) while skiping existing local files:

Scp All Files And Folders

Similarly, to upload all local files without replacing any remote file, first back up the file permissions of the remote destination folder. Then remove write-permission from all files in the remote destination folder. Finally, upload all local files, and then restore the saved file permissions.