Backing Up Your Backups

Like anyone who cares about the data that is on his or her computer, I keep backups. I don’t have the backups run automatically, as is the default in primary operating system, macOS. I also don’t do backups very often. But I do want to keep my data safe, encrypted, and off-site but also still easily accessible. As you might guess, backups for me are a complicated, manual affair. This is how that goes.

(Why don’t I run automatic backups? Well, a lot of the work that I do on my laptop is with local virtual machines running inside VMWare Fusion. Virtual disks attached to virtual machines get very big on your actual disk. Small changes to the data on the virtual disk usually results in huge changes to the underlying files that back those virtual disks that need to actually be backed up. Finally, backing up an in-use virtual disk is not conducive to a quality restoration. So before I do a backup I will stop my virtual machines and before manually initiating the backup. Then I go to sleep while the backup runs.)

To start, my primary computer is a laptop with a one terabyte hard disk that is usually about half full and whose contents churn fairly constantly. I also keep a second hard disk to store about two terabytes of assorted other files — mostly old photographs — that I want to archive but no longer need to be on my laptop. All of my backups from the laptop also go to the second hard disk. This disk is kept in my living room so that I may easily connect it to my laptop to access the assorted archived data and also to more easily run the backup process. That’s easy enough.

But after an apartment building next door to mine burned down back in 2009 I started keeping a copy of my backup disk in a fire safe in my apartment. After my fire safe got a crack in the casing I worried that it might not be as reliable as I expected it to be so I also started keeping a second copy of my backup disk in my desk at work. When I moved to Seattle and realized that an earthquake or volcano might wipe out both my apartment and my office in the same event, I also stared keeping a copy of my backups in the cloud.

I actually only have two files that I actually need backed up. That’s right: two. On my unencrypted external disk I have an encrypted sparse bundle image of my laptop’s Time Machine backups and I have an encrypted sparse bundle image containing about two terabytes of the digital detritus — mostly old photographs — collected from my twenty plus years using a computer. Actually, since they’re sparse bundle images those two files are really two directories containing approximately 223,000 files, but, uh, close enough.

So my primary backup disk contains those two sparse bundle images. After I perform a backup those two files change. The next step is to replicate those two sparse bundle images to my backup’s backups. How is this done? For the backup disks that I keep in my fire safe and in my office, this is easy: rsync. With both the original backup disk and the backup backup disk connected to my laptop, I open up Terminal and run this command:

cd /Volumes/original
rsync -rlptgoDzhOHi --stats --numeric-ids --delete-during *.sparsebundle /Volumes/backup

So that’s easy enough. It copies the sparse bundle images from one disk to another. I run the rsync twice, once to each backup backup disk, transport them to their respective locations, and I’ve got my two backups. The cloud backups are a little bit more complicated.

After researching a number of the backup options — S3, Glacier, DropBox, etc. — they just really weren’t feasible on cost. Using S3, for example, to back up 2TB of data would cost me about $45 a month, plus the cost of the data transfer and that starts to push $600 a year. Glacier is nearly impossible to use. DropBox doesn’t let you store more than 2TB per month on their less expensive professional plan and the option that lets you store unlimited data costs $60 per month or $720 per year.

But I did find an option that lets you store unlimited data and doesn’t cost an arm and a leg: Backblaze B2 Cloud Storage. My two terabytes is costing me $10 per month and there is no cost to transfer the data to their system and no cost to restore the data from their system. (And when I ran the upload from my office the only limitation on upload performance was my laptop’s 1Gbps network interface. I was able to push three to four hundred gigabytes every hour.)

Because I’m such a fan of rsync it turns out that there is a similar option for backing up to the cloud: rclone. After I set up my storage space on Backblaze I created an access key for my laptop and configured rclone with it’s incredibly simple “config” command and now I just run this command:

rclone --transfers 32 --progress sync /Volumes/storage/compy.sparsebundle/ b2-lockaby:lockaby/compy.sparsebundle/

“b2-lockaby” is the rclone nickname for my Backblaze bucket. Unfortunately, wildcards for matching files doesn’t work so I have to run this command twice: once for each “file” that I am backing up. Still, it’s trivial.

But there are a few catches to get to this point. First, my backup disks are ALL unencrypted but I require that I only store my data encrypted. That’s why my sparse bundles are encrypted. So when I connect my unencrypted disk I have to then open and mount my encrypted sparse bundles before I can use the data. When I do the rsync and the rclone I unmount the encrypted sparse bundles. For the sparse bundle image full of random data it’s easy to see how to set this up and how this works. But for the Time Machine backup this isn’t as obvious.

If you’re using macOS and you want to back up your hard disk you have two options. The first option is to connect an external disk to the computer and back up to that. If you tell macOS to encrypt the backup it will convert the disk to a FileVault disk. The second option is to connect your computer to a network disk such as one attached to an AirPort Express or AirPort Extreme. If you use the second option and you tell it to encrypt your backups then macOS will create an encrypted sparse bundle image on the network disk.

But Apple, in its shortsighted wisdom, has discontinued the AirPort line. As a result, being able to run a network backup seems like something that is going to cease being supported in the not-too-distant future. So I decided that directly attached Time Machine backups were going to be the future for me. But I obviously don’t want to convert my external disk to FileVault because then I won’t have encrypted sparse bundles that I can upload to the cloud.

The solution is actually pretty easy but not well documented. First, create an encrypted sparse bundle image on your external disk. Next, mount it. Then issue this command:

sudo tmutil setdestination /Volumes/{mounted-disk-image}

Now you’ll see Time Machine try to back up to that mounted image. When you finish doing your backup through Time Machine, unmount the image, rsync and rclone the sparse bundle, unmount the disk, and go back to your daily life.

Comments are closed.