Unison Sync Two Folders

Unison is used to store two replicas of a collection of files and directories on different hosts, modified separately, and then brought up to date by propagating the changes in each replica to the other. It can also be stored on different disks on the same host. Once past those two issues I managed to successfully get Unison to synchronize files between the Surface Pro and the Ubuntu system. Moving on to the MacBook Air—which I honestly suspected would be the easy step—I found Unison 2.48 crashed on macOS 10.15.

Unison Sync Two Folders

There are two types of digital nomads, those who have a single workhorse device and those who have several. This post covers the challenges faced by the latter type.

When I’m in the office I have a MacBook connected to a 34″ ultrawide monitor. This is where most of my work is done as it offers the best user experience. Lots of screen real estate, comfortable ergonomics, and perfect for long sessions. In the field, I have another MacBook that is a lot less enjoyable to use with it’s 13″ screen placed on random surfaces with the worst ergonomics and posture.

One of the challenges I face is synchronizing work in progress, ssh keys, and gpg keys between my devices. It’s been a problem for years and I’ve had several solutions, but none quite as elegant as Unison.

I discovered Unison when looking for an option to synchronize WordPress data across several machines serving a very high traffic site. Normally we’d use an EFS volume or deploy an NFS server, but both of those weren’t options for that client.

Unison is described as:

Unison is a file-synchronization tool for OSX, Unix, and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.

As advertised, I can sync my devices before leaving the office and take all of my work in progress, ssh keys, and gpg keys with me in the field. Once I return, with a single command I sync all of my progress in the field back to my office computer.

This process will differ depending on your operating system / Linux Distribution. Unison has to be installed on all machines that will be synchronized. The machines do not have to be running the same operating system, but you’ll have to set your paths correctly. More details in the comments of the Unison profile.

MacOS: I prefer to use Homebrew. If you do not have Homebrew setup, run the ruby command in a terminal to install it.

Debian/Ubuntu: Packages are available in both official repositories and the snap store.

2. UNISON PROFILE

Unison can be run completely from the command line and it also supports “profiles”. A profile is simply a set of parameters in a configuration file so that we don’t have to type out long error prone commands to sync our data.

In this scenario, the profile lives on the office computer in ~/.unison/.prf. The mobile laptop does not have a profile and all synchronization events are triggered on the office computer. An example profile, we’ll call it laptop,prf, it below and should be copied to ~/.unison/laptop.prf. Things you’ll want to modify in this profile are potentially:

  1. servercmd This is the path to unison on the laptop.
  2. root The first root entry points to the directory on the office machine containing the data we want to sync. We’ll specify exactly what to sync with path directives below. The second root entry is where our defined paths will be synchronized on the laptop. You’ll want to change the IP and path no doubt, but take careful note of the double forward slash (//), it’s not a typo.
  3. path The path directives specify what files or directories within the root that we want to copy. As mentioned, I want my repositories, gpg keys, passwords, and ssh key data.
  4. backupdir Specifies the directory in which the sync backups will be stored. This is relative so that it can be the same on mixed environment deployments (MacOS/Linux). It will be located in the root specified above.

3. SYNCHRONIZE YOUR DATA

Before we synchronize your data there are a few things to note.

  • Synchronization is always initiated by the office computer, even when the laptop has the most current data.
  • Due to our prefer=newer setting, any files that exist on both the office computer and laptop machine will be replaced by the newest copy.
  • In order to avoid potential overwrites, I suggest that the data only live on the office computer for the first synchronization, after that you can freely update content on either machine and be confident that Unison will update both of them with the newest files.
  • We tell unison to load the profile we created by specifying it (minus the prf extension) as the first argument.

Revenni is a Toronto based IT consulting firm specializing in Linux System Administration, Linux Consulting, Managed Linux Services, and 24×7 Emergency Linux Support.

Unison Sync Two Folders Published on 1 Jun 2020 · Filed in Explanation · 705 words (estimated 4 minutes to read)

I recently wrapped up an instance where I needed to use the Unison file synchronization application across Linux, macOS, and Windows. While Unison is available for all three platforms and does work across (and among) systems running all three operating systems, I did encounter a few interoperability issues while making it work. Here’s some information on these interoperability issues, and how I worked around them. (Hopefully this information will help someone else.)

Folders

The use case here is to keep a subset of directories in sync between a MacBook Air running macOS “Catalina” 10.15.5 and a Surface Pro 6 running Windows 10. A system running Ubuntu 18.04.4 acted as the “server”; each “client” system (the MacBook Air and the Surface Pro) would synchronize with the Ubuntu system. I’ve used a nearly-identical setup for several years to keep my systems synchronized.

One thing to know about Unison before I continue is that you need compatible versions of Unison on both systems in order for it to work. As I understand it, compatibility is not just based on version numbers, but also on the Ocaml version with which it was compiled.

Unison sync two folders windows 7

With that in mind, I already had a working setup using Unison 2.48 so I started there. Unison 2.48.4 was installed and running on the Ubuntu system, and I installed Unison 2.48.15 on the new MacBook Air. I’d used Unison 2.48.15 on macOS for quite a while, so I didn’t test the installation right away, instead moving on to the Surface Pro. From this page hosting Windows binaries for Unison, I downloaded a Unison 2.48.4 binary for Windows. Should be all set, right?

Unfortunately, I ran into a couple problems:

  • The Windows binary has an issue where it won’t recognize the preinstalled OpenSSH binary on Windows 10. So, I had to copy ssh.exe to the same directory as the Unison binary.
  • The Windows binary didn’t like paths with spaces in the name; no style of quoting seemed to help. The only workaround I could find was to use dir /X to get the auto-generated short name, and use that in the Unison profile.

Once past those two issues I managed to successfully get Unison to synchronize files between the Surface Pro and the Ubuntu system. Moving on to the MacBook Air—which I honestly suspected would be the easy step—I found Unison 2.48 crashed on macOS 10.15. Nothing would make it run without crashing.

Some continued research led me to find Windows and macOS builds of a newer version of Unison, version 2.51. The changelog referenced APFS support, which is what was being used on the MacBook Air. That should do it, right?

  • Unison 2.51 wouldn’t interoperate with the existing Unison 2.48 binary on the Ubuntu system. (Recall that I mentioned earlier that compatible versions of Unison were needed on both systems.)
  • There were no packaged versions of Unison 2.51 for Ubuntu.

Fortunately, cloning the GitHub repository and building from source was pretty straightforward. I changed the filename of the new version (I used unison-2.51.2) and changed the “servercmd” setting in the Unison preferences on both the Surface Pro and the MacBook Air. Success! I was able to run Unison to synchronize files between the Surface Pro, the Ubuntu system, and the MacBook air running macOS 10.15.

Lessons learned?

  1. Unison’s support on Windows for paths with spaces is tricky. Use dir /X to get the auto-generated short name and use that instead.
  2. On Windows 10, you’ll need to copy ssh.exe from C:WindowsSystem32OpenSSH to the directory where the Unison executable resides. Otherwise, Unison will be unable to initiate the SSH connection to the remote system.
  3. If you’re running macOS 10.15, be sure to use Unison 2.51. (This may apply to all systems running APFS, but I haven’t verified that yet.)
  4. Pre-compiled binaries for Unison 2.51 are available for both Windows and macOS, so that’s probably the best version to use. For Linux, it’s very likely you’ll need to build from source in order to get a version 2.51 binary.

Unison File Sync

Sync two folders mac

I hope this information is helpful to others who may need to get Unison working across multiple systems. If you have corrections, suggestions for improvement, or feedback on how I can improve this post, please contact me on Twitter. Thanks!

Unison Sync Two Folders Download

Metadata and Navigation

Be social and share this post!

Related Posts

  • Technology Short Take 1242 Mar 2020
  • Technology Short Take 10821 Dec 2018
  • Technology Short Take 9827 Apr 2018