Companies have not typically invested the resources to develop internal rate of delivery performance baselines that can be used to derive estimating templates. Therefore, industry data baselines of performance delivery rates are of significant value. The industry data points allow organizations to use these generic delivery rates as a means to "ballpark" their estimates. As they continue to develop an experience base of their own, they can transition from the use of industry data to use of their own data.
The desire for industry data is so great that many companies are willing to accept publicly available industry data at face value. Of growing concern is the fact that many providers and publishers of industry data have collected information that has not been validated, that is not current, or that is incomplete. To avoid any such pitfalls, the following criteria should be applied when obtaining industry data:
For what industry is the data representative, and what is the mix of data? Typically, industry data is a result of numerous data points that have been collected across a wide variety of industries. There is often no balance in the data; one or two industries represent the majority of data points. If they happen to coincide with your industry, then you are fortunate; otherwise, they may be of little or no significance.
What is the time period represented by the data? Vendors claiming to have thousands of projects in their database are not as quick to tell you that the data has been collected over an extended period of time. If you are looking for current data, you should realize that the sampling of current data will be relatively small and may not be representative of thousands of projects.
How valid is the data? Often data is collected from many sources. This limits the likelihood that the data has been validated. In addition, much of the data is of an empirical nature and, therefore, has some level of distortion.
Several opportunities currently exist for gathering, retrieving, and sharing of industry data. These organizations include The International Software Benchmarking Standard Group (ISBSG) and The Industry Software Practices Consortium (ISPC). Both organizations operate on the principle of a well-defined collection process that feeds a central repository and makes the data available for detailed access and comparison to industry best practices. The advantage of these industry databases is their accessibility of detailed data. The disadvantage is that they are relatively new and, therefore, have not yet collected a large population of data.