Tao Of Backup Wailing Wall Story

Even if you have a perfectly good backup, you may *not* be able to restore it.

OK, so it was my own fault. I fiddled around with my PGP keys and lost the private part of one main key (thought I had made a backup of the key, but only saved the public part). Only found out about it when I tried to uncrypt a file containing some personal e-mails months later.

No problem, I thought, my company does a backup of everything on a daily basis, and they keep montly snapshots for over a year, so let's just restore the keys from a backup before the erasure.

Problem was, the Computer Department only kept the *index* files for their backups for the past few weeks, so they were unable to exactly locate on which tape the file could be found. No problem either (or so I thought), they just needed to scan the relevant tapes (3 tapes, 40G each) for the file.

Well, I finally got a call that they were unable to find that particular file without investing further manpower, which would have to be paid by our department.

Turned out that there was only one tape drive available for both reading and writing the backups, and due to the long scanning time the sysop would have to come in rather early just to insert the tape so the scan would be finished by evening, when the daily backup was scheduled.

Needless to say, due to the "inofficial" status of my usage of PGP, no funding was available, so the key file is still not restored. I thus have lost some information that is not of interest for my company but still invaluable for me.

Lessons learned:

* _You_ are responsible for your personal data, don't assume that normal company procedures will guarantee the safety of your personal files. Do backups of critical data by copying files to a different file system.

* Have redundant backup hardware available, so backup and restore can be done independently, without one blocking the other (but also see next point).

* Check your backup *hardware* regularily (I'm missing this point in the Seven Heads of Backup Tao). Write backups with one tape drive and verify them on the other drive, so no hardware kink (like a misadjusted head) can render your backups unreadable (at a later time, after the drive got repaired). [We had that case with a CD writer: backups (or rather snapshots) of software release versions written with that writer were unreadable on a standard CD drive. We found out half a year later. Luckily, no damage was done.]

* And last but not least: keep old backups in a useful state. Save those indices on a different medium (e.g. CD Rom) with (semi-)direct access. Most "desasters" are localized, so it certainly is more economical to be able to do a partial restore rather quickly than to have to rescan all those gigabytes.

    Anonymous, Anonymous, Sat 26-Sep-1998 8:57am

Prev Wailing Wall Next

Back To The Sanctuary

Sanctuary Webmaster
Copyright © Ross Williams 1997. All rights reserved.