Intro and How-to of the Unix Backup Scheme at Caltech's GPS Department.

This document gives a quick intro into the backup scheme that is maintained in GPS. It also explains how to perform backups and restores of files and filesystems.

Review of the Backup Scheme

Backups of the Unix computers in GPS are performed using the Amanda software package. This is a freely available package. URL's of interest include:

The Amanda backup software designates two types of machines:

The master host is also a client machine, since it backs itself up. The master host contains one or more tape drives or a tape stacker and one or more holding disks reserved for holding the backups of filesystems before they are sent to tape. The holding disk is typically several gigabytes large.

When a backup starts, the master machine performs the following steps:

  1. Ask each machine for the amount of data to be backed up for each filesystem (partition).
  2. Determine the level of backup for each filesystem. If there are too many full backups of filesystems to do in a single amanda run, then amanda will delay the full dumps of some of the filesystems and perform incremental backups of these filesystems.
  3. Determine a schedule for the backups of the filesystems.
  4. Ask a few machines at a time to dump particular filesystems and send the dump over the network to the master host. The master host saves the dump to the hold disk.
  5. When a dump of a filesystem is complete, the master host transfers the dump from the holding disk to the tape.

Different configurations of hosts and filesystems to backup, types of backups, compression options, etc, are possible under Amanda. All of the options are saved under a particular "configuration." A configuration is known by a single label, such as DailySet1. The option and log files for a particular configuration are saved under directories named with the configuration's name. Under the directory are several important files. I only list here the files that you could commonly change.

DailySet1/amanda.conf             configuration options to amanda
DailySet1/chg-generic.conf        configuration for multiple tape drives
DailySet1/disklist                list of hosts and filesystems to backup

The Amanda programs and configuration files are owned by a particular user. All Amanda programs must be run by this user for successful operation.

GPS Specific Information

In GPS, kipuka is the master host. Kipuka is a SparcStation 5 running Solaris 2.5 and is located in 250 South Mudd. It has attached to it two Exabyte 8505XL's, one in an Andataco box and one in a Sun box. The devices are known as /dev/rmt/0 and /dev/rmt/1. Kipuka has two holding directories, /export/home1/amanda and /export/home2/amanda, both roughly 3 gigabytes large. Both are striped filesystems.

The amanda software lives under /opt/amanda and is owned by the amanda user (username amanda). When executing any amanda program, make sure that you are logged in on kipuka as amanda. The configuration name used for the daily GPS backups is DailySet1. The amanda user on kipuka has the following crontab entry which runs the backups at 12:01 AM everyday and ejects the tape when the backups are completed:
1 0 * * * /opt/amanda/libexec/cronjob Mail is sent to the users listed in the amanda.conf file when the backups are done.

Here's a picture of the directories and the information they contain:

.../bin                 executables for users
.../libexec             executables for Amanda software
.../config              directory containing different configurations
.../config/DailySet1    directory of configuration files for daily GPS backups
.../adm                 directory containing log files for configurations
.../adm/DailySet1       directory of log files for daily GPS backups
.../docs                directory of documentation for Amanda
.../man                 manual pages for Amanda

How-to: Backup Tasks

Daily Backups

For daily backups to be performed, new tapes must be inserted into the tape drives on kipuka. If you are using a brand new tape, the tape must be labeled with the following command:

amlabel DailySet1 DailySet1.???

Replace ??? with the next number in the sequence of tape labels. For example, at the time of writing, the last tape in the sequence is labeled as DailySet1.104, so the next tape to be labeled is DailySet1.105.

Check that the tape is loaded by doing:

mt -f /dev/rmt/0 stat

Continue doing this until you see this:

kipuka% mt -f /dev/rmt/0 stat
Exabyte EXB-8500 8mm tape drive:
   sense key(0x0)= No Additional Sense   residual= 0   retries= 0
   file no= 0   block no= 0

Now execute the following command:

amcheck DailySet1

Something similar to this should be seen:

kipuka% amcheck DailySet1
Amanda Tape Server Host Check
/export/home2/amanda: 3006679 KB disk space available, that's plenty.
NOTE: skipping tape-writeable test.
Tape DailySet1.047 label ok.
Server check took 14.817 seconds.
Amanda Backup Client Hosts Check
Client check: 27 hosts checked in 5.247 seconds, 0 problems found.
(brought to you by Amanda 2.3.0)

Any problems with down hosts, not enough space on the holding disk, or tape problems will be shown here.

Finally, rewind the tape:

mt -f /dev/rmt0 rew

Restoring a files or a filesystem

To restore a file or a filesystem, log in as amanda on kipuka. We then need to find out which tapes are necessary to do a complete restore. The amadmin program should be used as follows, replacing HOSTNAME with the proper host name:

amadmin DailySet1 info HOSTNAME

This shows information such as this:

Current info for earth /export/earth/u1:
  Stats: dump rates (kps), Full:   67.4,  68.5,  61.4
                    Incremental:    9.9,   5.5,  23.5
          compressed size, Full:  48.2%, 48.4%, 48.4%
                    Incremental:  48.7%, 36.9%, 28.2%
  Dumps: lev datestmp  tape              file  origK  compK secs
          0  19951225  DailySet1.045     128 886275 427197 6334
          1  19951226  DailySet1.046     125   2062   1005  101
          2  19951224  DailySet1.044      59   1490    551  100
          3  19951122  DailySet1.011     122  14909   9517  219

This shows on which tape and on what date a particular level dump was done. If we needed to restore earth's /export/earth/u1 partition, we would need tapes DailySet1.045 and DailySet1.046. Note that we would not need tapes DailySet1.044 and DailySet1.011, since these tapes were made before the last level 0 dump.

Insert tape DailySet1.045 into kipuka's tape drive and execute the following commands

cd /export/home2/amanda
amrestore -c /dev/rmt/0bn earth /export/earth/u1

This will leave a gzip'ed, dump formatted file in the current directory. Removing the -c option from amrestore will cause amrestore to uncompress the data before saving the dump file to disk. However, to reduce network traffic, we can uncompress the data on the remote host, earth in this case. Now log into earth as root and execute the following commands:

cd /export/earth/u1
/opt/amanda/libexec/zcat /net/kipuka/export/home2/amanda/earth*0 | restore xvf -

If you are restoring only a single file, then you can name the file in the restore command.

Blair Zajac
Last modified: Tue Apr 12 10:01:07 PDT 2005