IT Management Reference Guide

This is the second of a four-part case study on planning and conducting an operational recovery exercise. In part one I described how the project began, the decisions that the executives sponsors and I made at the outset, and the initial objectives and assumptions we developed. In this segment I describe how I effectively conducted weekly planning meetings, and a simulated walk-through of the exercise.

10.11.2 Weekly Planning Meetings

The executive sponsors from IT helped me determine the composition of the cross functional team who would plan and execute the recovery exercise. The kickoff meeting would be the key to a strong start so some extra planning went into it. I sent out a meeting invitation two weeks in advance and selected an optimal day, time and location to ensure a good turnout. These logical items may seem trivial but actually go a long way to gaining the all-important support and buy-in.

For meetings of this type I assign a scribe to take minutes and a timekeeper to keep us on pace. Figure 1 shows the agenda I used. I employed a similar agenda for all subsequent weekly planning meetings. Not all meeting facilitators prefer to use such a tight, time-oriented agenda, but I found this approach very helpful. When upwards of twenty highly-opinionated IT professionals meet to discuss details of a recovery exercise, a tightly enforced agenda becomes very valuable.

Figure 1 Agenda for Recovery Exercise Kickoff Meeting

Every week managers added a few more participants which usually brought fresh and meaningful perspective to items such as our objectives, assumptions and estimated time-frames. Spirited discussions from these perspectives usually led to numerous new action items to follow-up on each week. Table 1 shows a sample of types of action items captured and reviewed, and the format we used to track them.

Table 1

Table 2

Table 2

Simulated Walk-Through

Two weeks prior to the test, the recovery team and I scheduled and conducted a simulated walk-through. By now we had identified all the required steps needed to recover the 17 applications to our remote, out-of-state recovery data center. There were several reasons for conducting this walk-through:

• identify any missing steps
• ensure recovery steps were in the proper sequence
• validate that all predecessor and successor steps were correctly identified
• verify that the responsible person for each step was properly assigned
• revise any time estimates for steps involving start, end and duration times
• identify any additional issues that needed to be resolved prior to the exercise

The walk-through was well-attended and accomplished all of its objectives. All necessary tasks and assignments had been correctly identified, and only a few of the time estimates needed to be revised.

Next Steps

The final steps of preparation involved testing the special network and server procedures. These procedures would ensure that all branch office traffic generated during the exercise would route to the recovery data center, even though normal production at the primary data center would still be processing. Once these procedures tested successfully, we were ready for the actual exercise.

In part three of this series, I describe the results of the operational disaster recovery exercise conducted on Saturday, May 13, 2006.