- From Discovery to Verification
- Focused Investigations
- For EIA 731 Users
- Summary
This chapter is from the book
This chapter is from the book
This chapter is from the book
2.2 Focused Investigations
Large amounts of objective evidence are necessary due to the size of the CMMI model, the disciplines covered, and SCAMPI method rules. Efficient appraisals require effective data collection and management strategies. Focused investigations are used to track and prioritize appraisal team effort on the data that may still be needed for sufficient coverage of model practices within the appraisal scope. The appraisal team must continually keep in mind the following questions: What data do I have? What data do I still need? How am I going to collect the data? If the organization has prepared properly, the appraisal team's effort to verify the processes is much easier. How does the organization deal with the large amount of data required by the team to verify the processes?
Let's take a look at the amount of data to be collected and verified for an organization that is appraising three projects against the CMMI-SE/SW/IPPD/SS. There are 185 specific practices in this model (staged representation). Each specific practice requires at least one direct evidence data item. [1] Some practices may require more. This model has 25 process areas for maturity level 5, so the generic practice direct evidence may include at least one item of data for each GP for each process area. Table 2-1 summarizes the number of data items for each project.
Table 2-1. Data Items Required for Each Project (ML5)
|
Direct Evidence |
Indirect Evidence |
Total |
|
|
Specific practices |
185 |
185 |
370 |
|
Generic practices |
300 |
300 |
600 |
|
Total |
485 |
485 |
970 |
The appraisal team also looks for indirect evidence or affirmations for each practice. To reduce the risk of not satisfying the desired maturity level rating, let's assume that the organization and projects collect at least one indirect evidence data item for each practice. (Although this is not required by the SCAMPI method, the organization being appraised may feel safer by providing to the appraisal team indirect evidence data items for all practices.) This doubles the amount of data needed for the appraisal.
This adds up to a minimum of about 1,000 data items for a single project and 3,000 data items for three projects. In reality, there could be an even larger number needed to ensure that the appraisal team can verify each practice for each project. For example, some practices are compound sentences and a single data item may not satisfy the verification needs of the team. For instance, take GP 3.2: Collect work products, measures, measurement results, and improvement information derived from planning and performing the process to support the future use and improvement of the organization's processes and process assets. It might be difficult to create a process work product that shows all these expectations for a process area. More than likely, a measurement repository output will be needed as well as example work products in the process asset library or libraries, reports, and lessons learned. This GP is needed for every PA within the scope of the appraisal. Add these data items and their indirect evidence items and the number of data items to be collected for each project can easily approach 1,500 to 1,700. This results in 4,500 to 5,100 data items to be collected and verified for a threeproject appraisal at maturity level 5.
An undeniable conclusion: It is a significant task for the organization and its projects to manage the collection and presentation of this large number of data items.
However, it should be noted that these data items reflect the results of work performed by the project or organization. Thus, a highmaturity organization that would be striving for maturity level 5 does not have to create them—it just has to find them. This is the collection of work products that are the natural consequence of the organization and its projects following their defined processes. The task is to organize the data and map it to the model practices. While this is still a significant undertaking, one side effect of the effort may be the sharing of lessons learned across projects and across disciplines, which can support future process improvement efforts.
Please remember that requiring indirect artifacts for every practice is not required, and in many cases affirmations may be a much more efficient way to corroborate the direct artifacts.
Some organizations use hard copies of each data item and create indexes to the library. This was a common practice in CBA-IPI assessments. However, the larger amount of data required for CMMI is driving the need for automated libraries and online access to the data items. Some organizations have created databases that provide links to the files on servers or web sites that contain the data items. Others have created web sites that control the access to the same types of information. There are also tools provided by software development companies and consulting organizations to be used by their clients.