A small scale backup strategy

18 November 2009

Everybody knows that making backups is very important; imagine for example loosing your entire archive of family photos. Regrettably, coming up with a good backup solution is rather difficult. For example, when you make sure you've got every piece of valuable data backupped to a CD or a DVD, you might be surpised to find that your DVD went corrupt by the time you need it. Or when you make backups to an external hard drive which is permanently attached to you computer might be partially deleted by a computer virus. Also, as both your hard drive containing the original data an the one holding the backup are connected to the same machine, chances are they are both destroyed when a lightning stroke hits your house. Or what will happen if a burglar takes your PC?

These are only a few examples of what can happen with your precious data. I have always tried to secure my personal data, but it took me a long time to come up with solution I find exceptable. I didn't find much good solutions on the internet, so I decided to publish my personal backup strategy.

Summary

In short, it comes down to copying your original hard disk to two external disks alternatingly. One of both disks must be stored in a different building than your original data; in other words: it must be taken "off-site". MD5 checksums of all files are stored, so you can be warned when files are changed unwantedly, for example because of degradation of your backup medium.

Some rules to follow

• Use 3 hard disks: 1 original and 2 backups. The disk holding the original can be built into the system; the two backups must be external hard drives that can easily be taken off-site. Use the smallest disk as the original.
• Keep all data that needs to be backed up on a single disk (the "original"). This simplifies the configuration of the backup system.
• Only connect the backup disks to the system to make backups. Keep them disconnected the rest of the time.
• Always keep one of both backup disks off-site. Make sure you never have both disks in the same building at the same time.
• If necessary, encrypt both backup disks using TrueCrypt. This makes it easier to store them off-site.
• Make a backup every once in a while; for example each month.

Backup steps

The following steps have to be repeated every time you want to make a backup:

• Check the MD5 checksums of both the original and the backup disk (for example using ExactFile). This detects medium degeneration.
• Use a backup tool to copy the files from the original to the backup disk (for example using Second Copy). Make sure old versions of changed files are kept for a while.
• Eliminate duplicates of big files between the backed up latest version and the older versions (for example using Duplicate Cleaner). Only delete from the older versions directory.
• Create MD5 checksums for both the original and the backup disk. Store the checksum file on the disk they describe. Only include the newest versions of the files.
• Compare the newly generated checksum files between the original and the backup (for example using Beyond Compare). This ensures the backup was made correctly.
• Take the fresh backup off-site and bring the other backup disk back home. This way, there is a new version on-site (on the original disk) and one off-site.

Which problems does this solve?

• Current hard drives should provide sufficient storage space to store all valuable data. Only movies can fill this amount of disk space.
• As all data is stored on a single disk, there is no need for maintaining an index of what is stored where. The directory structure serves as documentation.
• The backup process requires only little effort, and it can be run at night.
• If one of the hard disks crashes, there are still 2 other copies.
• If one of the disks slowly degenerates, you are warned because some MD5 checksums failures will occur. When this happens, it is time to replace the disk.
• If a large file has small errors on all 3 copies, it might still be possible to reconstruct the original file (unless the errors are at the same location in the file).
• In case of location dependent storage medium loss (fire, flood, loss, theft, electrical surge, ...), there is always a copy on the other location.
• In case of softwarematic modification and deletion (files modified/deleted by mistake, computer virus,
  ...), the file can be recovered from the backup disks. Because the disks are disconnected, the virus cannot mess up the backups.
• Bugs or malconfiguration of backup software will be detected when comparing the MD5 checksum files.
• Because previous versions of the files are stored, deleted files can still be recovered, even if the deletion went unnoticed for a long time.
• Because the backup disks are encrypted, they can be safely stored at the office.
• When the password of the backup disks is forgotten, there is always the original copy. However, this situation should be avoided.
• The disk to be used for the next backup is always ready at hand (that is not off-site), so it is possible to immediately start making the backup when it springs to mind. The freshly made backup disk needs to be taken off-site only afterwards, so it requires no planning.