IT Management Reference Guide

This case study describes my recent efforts to access a production acceptance process at large, well-established aerospace firm. The company is over fifty years old and enjoys a reputation for researching, developing, and applying cutting-edge technology for both the military and commercial sectors. At the time of my assignment it employed 80,000 workers of whom 400 resided in IT. The IT platform environment of its main corporate computer center consisted primarily of four huge mainframes, with the same number of midrange computers and servers and approximately 1200 desktops.

The IT operations department of this company had a well-established production services function that ran 350 production applications daily, slightly more during month-end processing, and deployed on average two new production applications per month. There was no quality assurance group at this company although they did have the beginnings of a formal change management and production acceptance process.

Case Study on Production Acceptance: Production Services

The Production Services department was staffed by two very competent individuals who thoroughly knew the ins and out of running virtually every production application in the company, though little of it was documented. They were very technically knowledgeable as was most all of the staff in IT. This reflected part of the company's mission to develop highly technical expertise throughout the enterprise. Another part of the company's mission was to dedicate every department to continuous process improvement. The production services function was still very manually oriented and consequently somewhat inefficient. No automated scheduling systems were in place here at this time, but the company was willing to try new techniques and to technologies to improve their processes.

The Production Services group was also very segregated from other processes such as change and problem management. There was only the start of a production acceptance process which was not tied to production services at all. This segregation occasionally strained communications between operations and applications development. The fact that they were 25 miles apart sometimes added to the lack of effective face-to-face meetings.

Operations did a good job of collecting meaningful metrics such as outages, abnormal terminations, reruns, reprints, and reports delivered ontime. There was an inconsistent emphasis on how often or how deeply there metrics should be analyzed which sometimes undermined their usefulness.

Case Study on Production Acceptance: Strengths and Weaknesses

To summarize this company's strengths, they were willing to try new techniques and new technologies, committed to continuous process improvement, hired and developed a technically competent staff, and were willing to collect meaningful metrics. To summarize their weaknesses, they tended to not interact with members of other IT staffs, provide little documented training, did not always have effective communications with the development group due in part to a 25 mile separation, and did not always analyze the metrics they collected.

Eventually, the operations department implemented a more formal production acceptance process. One of the most important lessons we learned was to ensure the operations department was involved very early with a new application project. This helps ensure that the appropriate operation's group provide or receive the proper resources, capacity, documentation, and training required for a successful deployment. The other important lesson we learned was that the other infrastructure support groups such as network services, the help desk, storage management, and desktop applications need to provide their full support to the production services function. Because this function had worked in an isolated manner in the past, other infrastructure support groups were initially reluctant to support it. With improved processes, automation, and increased communication, the problem of lack of support was resolved.

Case Study on Production Acceptance: Process Assessment

Over the years, many clients have asked me for a quick and simple method to evaluate the quality, efficiency, and effectiveness of their production acceptance processes. In response to these requests, I developed the production acceptance assessment worksheet shown in Table 1. Process owners and their managers collaborate with other appropriate individuals to fill out this form. Along the left-hand column are 10 categories of characteristics about a process. The degree to which each of these characteristics is put to use in designing and managing a process is a good measure of its relative robustness.

Table 1 Assessment Worksheet for Case Study Company

View Table

The categories that assess the overall quality of a process are executive support, process owner, and process documentation. Categories assessing the overall efficiency of a process consist of supplier involvement, process metrics, process integration, and streamlining/automation. The categories used to assess effectiveness include customer involvement, service metrics, and the training of staff.

The evaluation of each category is a very simple procedure. The relative degree to which the characteristics within each category are present and being used is rated on a scale of 0 to 3, with 0 indicating no presence and 4 representing a large presence of the characteristic. For example, at this particular company the executive sponsor for the production acceptance process demonstrated some initial support for the process by carefully selecting and coaching the process owner. However, over time this same executive showed only mild interest in engaging al of the necessary development managers and staffs in the process. I consequently rated the overall degree to which this executive showed support for this process as small and gave it a 2 on the scale of 0 to 3. On the other hand, the process owner was extremely knowledgeable on all of the critical applications and their deployments resulting in us rating category a 3.

I similarly rated each of the categories as shown in Table 1. Obviously, a single column could be used record the ratings of each category; however, if we format separate columns for each of the four possible scores, categories scoring the lowest and highest ratings stand out visually. I have filled in sample responses for each of the categories to show how the entire assessment might work. The next step is to sum the numerical scores within each column. For this company, this totals to 1 + 3 + 4 + 3 = 11. This total is then divided by the maximum possible rating of 30 for an assessment score of 35%.

Apart from its obvious value of quantifying areas of strength and weakness for a given process, this rating provides two other significant benefits to an infrastructure. One is that it serves as a baseline benchmark from which future process refinements can be quantitatively measured and compared. The second is that the score of this particular process can be compared to those of other infrastructure processes to determine which ones need most attention.