Linux Backups For Real People, Part 2

By: Carla Schroder
Thursday, November 8, 2007 10:18:54 AM EST
URL: http://www.linuxplanet.com/linuxplanet/tutorials/6435/1/

Single-User Backups

Last week we got our backup hardware in order, so today we're going into detail on backing up our data to a locally-attached backup device. We'll learn how to configure which files to backup, and create an easy one-word-command backup.

I like to divide backups into two categories: system and data. This little series is about data backups; check out MondoRescue for easy system cloning and disaster recovery.

We're not going to use any fancy, complicated specialized backup applications, but plain old reliable efficient rsync. rsync transfers only changes, so after your first run subsequent backups are very fast. It creates ordinary Linux directory and file structures, so restoring files is done via your usual favorite file copying method, rather than needing special backup software and weirdo commands.

Your backup drive should be a minimum of two times larger than the total of the files you want to backup. Graphical file managers will tell you how large your directories are, though I like the good old du (disk used) and df (disk free) commands because they're fast, and they work the same way on all Linuxes. Just fire up a terminal and run this command to see the used and free space on your local filesystems:

$ df -hlx tmpfs
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda1              14G  2.4G   11G  18% /
/dev/sda1              31G  9.3G   21G  32% /home
/dev/hda2             4.5G  543M  3.8G  13% /var

You'll need du for individual files and directories, as this example for my finances directory shows:

$ du -hs finances
8.2M    finances

Leave off the -s to see the files in the directory displayed, and their sizes.

Making a Test Backup

Using the backup device naming examples from last week's article, let's make a test backup to a USB drive of any kind. Plug it in, and then copy a couple of directories to it using this command. Remember, in part 1 we formatted the backup drive in FAT16/32, so the rsync options are special to the FAT filesystems. In this example I backup my finances and configs directories, using the full absolute path names:

$ rsync -rlvt --modify-window=1 ~/finances ~/configs /media/BACKUP1

The tilde, ~, is a shortcut for home directory. You'll see filenames whiz by, and then a summary:

sent 8306732 bytes  received 1270 bytes  3323200.80 bytes/sec

total size is 8301576  speedup is 1.00

Now open /media/BACKUP1 and admire your backed-up files. They look like normal files, which they are, which you can view and copy like any normal Linux files.

Mind your trailing slashes! Notice there aren't any, so I'll end up with /media/BACKUP1/finances and /media/BACKUP1/configs. If I named finances/ and configs/ instead, then I would get /media/BACKUP1 without the two directories, but only the files that were in them.

Now try a cool thing: hit the Up arrow to display your rsync command again and hit enter to re-run it. You'll see something like this:

$ rsync -rlvt --modify-window=1 ~/finances ~/configs /media/BACKUP1
sent 1582 bytes  received 20 bytes  3204.00 bytes/sec
total size is 8301576  speedup is 5182.0

Notice the difference in the "sent" values. This shows that your rsync incantation worked correctly and transferred only changes on the second run.

The -r switch means recursive; -v is verbose, -l copies symlinks, -t preserves the modified times, and --modify-window=1 allows for a 1-second difference in the modified times between the original and copied files. The last three options are necessary hacks for FAT16/32. If you leave out the -t switch rsync will re-copy your files instead of transferring only changes. FAT16/32 represents times with up to two seconds' resolution, so you need the --modify-window option to allow for this.

rsync has a --dry-run option which you should always use when you're not sure if you have it right. This shows you what will happen without actually doing anything.

Refining Your Rsync Incantation

rsync has a lot of options, which you'll see in man rsync. Here are some of the more useful options:

-- delete
When you delete files from your system, using this option will also delete them from your backup drive.

--exclude-from=[filename]
You can create a file that specifies filetypes that you don't want backed up, putting one item per line like this:

*.tmp
*.temp
*.bak

--include-from=[filename]
You may use this option to list directories to include, but it must be done in the exactly correct way, or it won't work, and then you'll join the legions of sad, lost people roaming Google trying to find answers. I'm backing up /home/carla/finances and /home/carla/configs, so I make these entries in my rsync-includes file; the double asterisks ensure that all sub-directories will also be copied:

+ /finances
+ /finances/**
+ /configs
+ /configs/**
- *

$ rsync -rlvt --modify-window=1 --delete 
--include-from=/home/carla/rsync-includes /home/carla/*  /media/BACKUP1 

Throw in another -v or two for debugging if you run into trouble. There are more ways to use --include-from and --exclude-from, but that's as far as we're going today.

That's all a bit much to type a lot, so one good shortcut is to create a Bash alias:

$ alias backup1='rsync -rlvt --modify-window=1 --delete
 --include-from=/home/carla/rsync-includes /home/carla/*  /media/BACKUP1'

So now all you do is type backup1 to run your backup. You can run it as often as you like; for example, if you're working on an important document there's no reason you can't run your backup command as often as your "save" command.

Come back next week for Part 3 to learn how to do easy network backups, how to automate your backups, and how to create nice desktop icons for one-click backups.

Resources

• man 1 rsync
• man 1 df
• man 1 du
• man 1 bash to learn about alias