Automation

Unison file synchronisation in Mac OS X

Posted in Automation, IT on July 18th, 2012 by Matt – Comments Off on Unison file synchronisation in Mac OS X

Currently I use Unison over SSH to sync my documents and some specific system files.

I have been using Unison for over seven years and it’s great. Unison works a bit like rsync in that although initial sync may take a long time afterwards only differences are transfered. Conflict resolution is handled well with the user able to override suggestions by skipping or forcing a specific direction of sync i.e. local > remote or remote > local.

Another advantage of the way you configure Unison is that server information i.e root sync directories of local and remote hosts can defined in a separate file to the paths to be synced or ignored. This means that bi-directioal syncing is possible with the same configuration file. This allows Unison to sync it’s own configurations, which is very useful as an identical sync set can be initiated from either side. Similarly you can split your synchronisation into parts via different profiles so not all files are synced all the time. For example, do you really need to sync all your old university files which you don’t change them very often any more? This makes scanning quicker.

The other advantage is syncing is done over SSH, so if you setup public key authentication then only one port and no password is needed. This also opens up possibilities beyond the LAN to secure syncing over the internet via NAT.

Finally, although here I discuss a homogenous all mac LAN Unison is cross platform. Install the client, on windows or linux and happily sync all data. This is also a good way to sync data off accounts with restricted permissions as you all you need to do be able to do is initialise an ssh connection i.e. be a client not a server, and tunnel everything else.

Finally although the GUI is nice for monitoring, resolving conflicts or forcing overwrites the CLI can easily handle simple synchronisation sinarions, and has options for automated conflict resolution. This allows specific sync profiles to be initiated by scripts or by cron.

Currently my common file on A is:

root = /Users/
root = ssh://@//Users/

wheras on B it is:
root = /Users/
root = ssh://@//Users/

Syncing of more than two devices us undertaken by syncing to one central device sync. Thus if B is the central device full synchronisation is achieved by a sequential sync:

A <-> B then B <-> C then A <-> B

After this you just need to sync each time you switch computer i.e. A <-> B or B <-> C. If you edit the same file on multiple computers a conflict will arise and an overreide direction can be specificed i.e. A > B or B > A.

My current setup sync my whole iPhoto Library (120Gb) and my iTunes library (100 GB). Unless I edit/add lots of pictures then syncing only takes afew minutes. For example when I upgraded all my iPhoto thumbnails for the retina display of my new laptop this was just as few GB of syncing.

My regular profile is:

include common

# Paths to synchronise
path = Books
path = Desktop
path = Documents
path = Movies
path = Music
path = Pictures
path = Sites
path = .bin

path = .tcshrc
path = Library/Application Support/Unison
path = Library/Application Support/myTracks
path = Library/Temperature

# Paths and files to ignore
ignore = Name .DS_Store
ignore = Name *.vmwarevm

ignore = Path Desktop/*.avi
ignore = Path Desktop/*.mov
ignore = Path Desktop/*.download
ignore = Path Documents/Microsoft User Data
ignore = Path Documents/Academic
ignore = Path Pictures/Webcam/archives
ignore = Path Sites/logs
ignore = Path Sites/pfarrplatz.jpg

ignore = Path Library/Application Support/Unison/ar*
ignore = Path Library/Application Support/Unison/fp*
ignore = Path Library/Application Support/Unison/unison.db
ignore = Path Library/Application Support/Unison/common

ignore = Path Library/Temperature/current-outside.txt
ignore = Path Library/Temperature/0-min.log
ignore = Path Library/Temperature/5-tmp-day.log

logfile = Library/Logs/Unison.log

My irregular profile is:
include common

# Paths to synchronise
path = Documents/Academic
path = Documents/Work

path = Library/Application Support/Unison

# Paths and files to ignore
ignore = Name .DS_Store
ignore = Name *.vmwarevm

ignore = Path Library/Application Support/Unison/ar*
ignore = Path Library/Application Support/Unison/fp*
ignore = Path Library/Application Support/Unison/unison.db
ignore = Path Library/Application Support/Unison/common

logfile = Library/Logs/Unison.log