Every year that goes by, I think more and more about backup strategies for our home computers. As more of our lives become digitized, it's increasingly important to insure that these memories are preserved. And I'm not talking about backing up a single computer here and there. I mean a comprehensive backup strategy that encompasses all the devices in a household with multiple levels of redundancy.

Two years ago, I would have said "The Cloud" is the answer to everything. But as I quickly learned, it's impractical to backup anything more than a few gigs of data online. Network speeds are not yet capable of transferring hundreds of gigs in any reasonable timeframe.

I recently revisited this question of backups, and here's my current thinking. Hopefully this is something that will scale for years to come.

Overview

Here's an overview of what our backup system looks like:

The key to this system is redundancy. I've tried to build in an adequate level of redundancy so that the data is replicated multiple times across different locations.

Network Attached Storage

I have a ReadyNas NV+ to consolidate all our data. Photos, music, movies, documents, all of it goes on the 2TB storage array. The ReadyNas uses Netgear's proprietary X-RAID technology to offer redundancy should any single drive in the array fail. I split the drive into three partitions:

  1. Personal - Stores personal photos/videos, documents, backups from other computers, and a personal Mercurial source control system (For any code I play around with). This currently comes out to about 30 gigs (mostly photos). This entire partition is backed up to Amazon S3.
  2. Media - Stores mp3s, movies, and any large format media files. The files here are too large to store online.
  3. Backups - Stores Time Machine backups of other laptops.

Backing up Home Computers

At regular intervals, I run a simple rsync script to backup all the home computers to the ReadyNas' "personal" partition. I only backup the user's home folder, which is a considerable space saver (about 10gigs vs. 70gigs for the entire computer).

(Tools such as SuperDuper! will backup an entire computer to a bootable drive, but frankly, this seems overkill to me. Most of the software I use is free or easily replaceable. If a laptop were to fail, the applications would not be missed; I even enjoy a fresh install every few years. The data is what's most important, and that is what gets backed up to the ReadyNas.)

Full computer back ups do have their benefits, which is why I still run Time Machine. I like how Time Machine offers versioned files in an easy-to-use interface. The Time Machine backups go to a special partition on the ReadyNas.

Devices like digital cameras, flip cam, and iPhones are also saved to the ReadyNas' "personal" partition.

Backing up the ReadyNas

Backing up all our devices to the ReadyNas offers decent protection, but since the ReadyNas is always connected to the network, it is vulnerable to accidental file deletion and human error. So the entire ReadyNas is itself backed up to an external USB drive (again using rsync). This doesn't have to happen often, but it should happen regularly. I try to do it once a month, around the same time I do the monthly bills.

(Backup experts say you should store your external backup somewhere offsite, so its protected from fire, theft, etc. I'd like to say that I store my backup offsite, but the truth is, it just doesn't happen. Lugging the drive back and forth between my office is just a pain. Maybe a fireproof safe would be a worthy investment.)

Backing up to Amazon S3

Even with the ReadyNas, even with the external drive, I have a nagging sense of paranoia about my most valuable data. Wedding photos, photos of my newborn son, these are all files that are irreplaceable. These files, along with the other files in the "personal" partition, get backed up to Amazon S3. There is a cost associated with Amazon S3, I estimate less than $5 a month. Its a small price to pay for peace of mind. However this cost will grow, and its something I'll need to keep an eye on and adjust with time.

I don't backup the "media" or "backup" partitions to S3. These partitions are over 100gigs and would prove costly and time-consuming to store on S3.

Conclusion

With this backup strategy in place, any piece of data exists in at least four times: once at the original location, once on the ReadyNas, once in Time Machine (on the ReadyNas) and once on the external USB drive. The most important pieces of data are backed up one more time to Amazon S3. Most of these steps can be automated, the only additional work is plugging and unplugging an external USB drive once a month. Will this scale? I hope to explore that and other details of this backup system in future posts.