Home > Articles > Software Development & Management

  • Print
  • + Share This
Like this article? We recommend

A Cautionary Tale

Having spent lots of money on a backup solution is far from a guarantee of security either. Witness this cautionary tale. I personally know of a company where the security of the enterprise backups was turned over to a large online storage system with terabyte tape capacity in a single large unit where tapes lived never to be touched by human hands. Centralized backup made sense with their many different platforms and servers, and it did make the job of backups so much easier; no more tapes to change—the mysterious black box in the corner does it all! But belief that "expensive" somehow meant the same as "foolproof" established the conditions for disaster when one of the servers crashed.

The backup unit was functioning perfectly, as was the software, but backups had been done only twice in a six-week period. One test backup was initiated when the system was first put it place, and another was programmed for unattended backups. Unfortunately, the schedule was not set to repeat—it was a "one only" backup, and no one bothered to check logs or the validity of their backups. To make matters worse, the backup system still required that bootable system backups be used to re-create the initial system before the restore (of what was now six-week-old data) could take place. Things like device configurations and filesystem sizes were not part of what this system archived. The users failed to take that into consideration—after all, the solution they had purchased cost a lot of money. Surely, it did everything. Those bootable tapes, by the way, were thought no longer necessary and had been disposed of or reused.

What can we learn here? One lesson is as follows: Tens of thousands of dollars in sophisticated software and hardware will never completely replace human intervention and regular checks of your logs. Remember, those backups are your corporate lifeline. Here's another: If you are going to move from one backup system to another that is incompatible with what you now run, make sure that you have at least two backups using your old technology and that you store those tapes more or less permanently. It is also wise to keep the programs necessary for recovery of those final backups live and on your system.

This brings up the question of the comfort and peace of mind any technology or software can offer. Personally, I still have a fondness for tapes I can touch. When I take my tape cartridge from my SCSI drive and hold it in my hand, I know what's on that tape. I mounted the tape myself. I backed up the data with scripts or tools I personally loaded. I even verify the occasional tape myself. Consequently, I know the data is good. If I can't actually touch the tape with my system on it, (as in the large storage unit that never gets opened), I don't feel quite as confident. With removable tape cartridges, I can easily label them and store them offsite for greater security (not too far away; you want to get at those tapes quickly, should the unthinkable occur). I will admit that those feelings come from years of being a hands-on systems administrator. Hard disks crash. Power supplies fail. Systems overheat. I don't completely trust any technology despite my appreciation of said technology. Sometimes, simpler is better. Sadly, it's just not always practical.

Backups are a dirty job that no one really wants to do. In these days of high-capacity tape drives, that job can be greatly enhanced with the proper software tools. Remember the lessons. Do it right, and you may even have fun doing it.

  • + Share This
  • 🔖 Save To Your Account