With Contributions by Capers Jones, Howard Rubin, David Garmus, Lawrence Putnam, and Elizabeth Clark
The accurate, quantitative measurement of software quality and process performance is rapidly becoming an essential part of competition in the ever-tightening software marketplace. Software metrics provide insights into productivity and quality gains from improvements in skill, technology, and development methodology. An effective metrics program helps practitioners assemble the best team, select the optimal development methodology, and enhance the quality of a software product. In short, metrics enable software developers to pursue proven, successful strategies, and to change course when metrics point to less-than-optimum quality or productivity.
Written by the world's leading authorities in the field, IT Measurement showcases state-of-the-art in software metrics and provides the practical knowledge that practitioners need in order to take full advantage of software metrics technology. The book's collected articles offer important perspectives on the role of metrics in the development process, and show how metrics directly enhance software quality and output efficiency.
The book explores several vital areas, including Function Point Analysis, project estimation and management, outsourcing, statistical process control, and more. These articles range from basic theory to the sophisticated application of metrics. Specific topics covered include:
Enlightening and pragmatic, IT Measurement will help you gain a deeper understanding of software metrics and the ability to apply concrete measures in order to objectively evaluate and more finely shape your software development program.
I. INSIGHTS FROM INDUSTRY EXPERTS.1.The expanding roles of function point metrics.
Expanded Uses for the Function Point Metric.
1975: Function Points Invented Within IBM.
1978: IBM Announces Function Points to the Software Industry.
1980: Function Points and Programming Language Evaluation.
1985: Function Points and Software Cost Estimating.
1986: Formalized Function Point Training.
1987: Function Points and Software Quality.
1988: Function Points and Software Benchmarks.
1989: Function Points and Software Portfolio Analysis.
1990: Function Points and Tool Analysis.
1991: Function Points and Make Versus Buy Analysis.
1992: Function Points and International Industry Comparisons.
1993: Function Points and Outsource Analysis.
1994: Function Points and Business Process Reengineering.
1994: Function Points and the Software Engineering Institute Capability Maturity Model.
1995: Function Points and Taxation of Software Assets.
1995: Function Points and Outsource Litigation.
1996: Function Points and Estimating Rules of Thumb.
1997: Function Points and Mass Updates such as the Year 2000 Projects.
1997: Function Points and Enterprise Resource Planning Applications.
1998: Function Points and Web-Based Applications.
1999: Function Points and the Euro.
2000: Function Points and the Earned-Value Approach.
2001: Function Points and the Balanced Scorecard Approach.
Optimizing Function Point Value.
Function Points and Software Requirements.
Ratios of Function Point Counting Specialists.
In-House Counts Versus Consultant Function Point Counts.
Potential Expansion of Function Point Metrics.
Improved Guidelines for Embedded and Real-time Software.
The Need for Data Point Metrics.
The Need for Service Point Metrics.
The Need for Engineering Point Metrics.
The Need for COTS Point Metrics.
The Need for Value Point Metrics.
The Need for Risk Point Metrics.
Summary and Conclusions.
Biography.2. IT Organization, Benchmark Thyself.Michael Mah
Metrics That Matter.
Compared to What? A Frame of Reference.
Interpreting the Metrics.
Collecting the Data: You Can Be Your Own NCHS.
Managing the Data.
Establish a Routine.
Organize What You Have Learned.
Present What You Have Collected.
The Driving Pressure: Deadlines.
Obstacles to Acquiring IT Metrics Information.
What Projects to Use.
Old Data Versus New.
Do Not Play the Ratio Game.
A Graphical Analysis.
What It All Means.
Biography.3. The Core of Software Planning.Lawrence H. Putnam and Ware Myers
Measuring the Core Metrics.
Measuring the Product.
Measuring Process Productivity.
Measuring the Time.
Measuring the Effort.
Measurement Seeks Results.
Estimating with the Core Metrics.
Bidding with the Core Metrics.
Control with the Core Metrics.
Replan with Revised Metrics.
Biography.4. Work Output Measurement: IT Work Units.Howard Rubin
Work Output Measurement.
Normalization and the Work Unit.
Initial Observations on ITWUs.
ITWU Example from MCI Systemhouse.
Evolutionary Development of a Purer Work Unit Metric.
II. MEASUREMENT PROGRAM APPROACHES.5. EDS Brazil Metrics Program: Measuring for Improvement.
Why a Metrics Program and Why It Must Be Formalized.
Metrics Program: Organizational Structure.
Metrics Program Processes.
Why Some Metrics Programs Fail.
EDS Brazil and the Corporate Metrics and Estimating Program.
Biography.6. Measurement Program Implementation Approaches.Lori Holmes
Step 1: Defining Goals and Initiatives.
Step 2: Defining Measures.
Step 3: Defining Data Collection.
Data Collection Points.
Data Collection Responsibilities.
Data Collection Vehicles.
Step 4: Defining Reports.
Step 5: Implementing the Measurement Process.
Process and Methods Development.
Biography.7. Benchmarking.Arlene F. Minkiewicz
What Is Benchmarking?
Getting Started with Benchmarking.
Analyzing What Has Been Learned.
Biography.8. A Data Definition Framework for Defining Software Measurements.James Rozum
Uses of Data Definition Frameworks.
Data Modeling and Terminology.
Data Definition Columns.
Definition Column Descriptions.
Example of Data Definition Columns.
Data Manipulation Columns.
Putting It All Together.
An Example from Practice.
Complete Effort (Staff Hours) DDF.
Biography.9. Cheaper, Better, Faster: A Measurement Program That Works.Janet Russac
Metrics: Concept, Benefits, and Uses.
Metrics Selection Criteria.
The Measurement Team.
Data Analysis Process.
Organizational Metrics Reporting.
III. FUNCTION POINTS AS PART OF A MEASUREMENT PROGRAM.10. How and When Can Functional Size Fit with a Measurement Program?
Software Measurement Steps.
Planning the Measurement Program.
Which Metrics to Choose.
Pilot the Appropriate Target Metrics.
Critical Success Factors.
Biography.11. An Early Estimation of Software Reliability Based on the Size Estimation and the Software Process Assessment of Large Telecom Systems.Alex Lubashevsky
Software Reliability Estimation.
The Role of Size and Complexity.
Early Size and Reliability Estimation.
Biography.12. Establishing Central Support for Software Sizing Activities in a Large Organization.Valerie Marthaler and Steve Keim
ISO standard 14143-1.
Methods and Tools.
Function Point Analysis.
Model-Based Estimating Tools.
Function Point Analysis Repository Tools.
SLOC Counting Tool.
Recommended Organization Structure.
Central Sizing Support Team Responsibilities.
Central Point of Contact.
Recommend Sizing Approaches.
Training and Mentoring.
Establishing and Maintaining Asset and Project Sizes.
Validating Asset and Project Sizes.
Providing Issue Resolution.
Communicating Organizational and Industry Standards and Guidelines.
Reporting Back to the Organization and Leadership.
Biographies.13. Function Points as Part of a Measurement Program.Pam Morris
Managing Project Development.
FPA Uses and Benefits in Project Planning.
FPA Uses and Benefits in Project Construction.
FPA Uses and Benefits after Software Implementation.
Customizing Packaged Software.
Estimating Package Implementations.
FPA Infrastructure and Resources.
Roles and Responsibilities.
Selecting Software to Count.
Activities That Do Not Deliver Function Points.
Standards for Recording Function Point Counts.
Standards for Resolving Count Issues.
Standards for Reviewing Counts.
Resources and Cost.
Cost of Implementation.
IV. USING METRICS TO MANAGE PROJECTS.14. Tracking Software Progress.
Tracking Progress: Criteria for Effective Measures.
Activity-Based Measures of Progress.
Detailed Activity-Based Measure.
Product-Based Measures of Progress.
Comparing Activity-Based and Product-Based Measures of Progress.
Biography.15. Effectively Utilizing Software Metrics: Project Metrics.Daniel D. Galorath
Strategic versus Tactical.
The Project Framework.
Function Points and Sizing.
Earned Value Management.
Product Defect Tracking: Managing with Metrics.
Biography.16. Project Metrics Using Effort Metrics for Tracking.Magnus Hoeglund
Why Measure and Analyze?
Avoiding the Pitfalls.
Effort: What Is It?
Active Planning and Tracking.
Tracking on Different Levels.
Biography.17. Using Project Metrics to More Efficiently Manage Projects.Steven M. Woodward
New Development Project Management.
Enhancement Project Management.
Project Duration Days.
Lines of Code.
Practical Real-Life Application of Project Metrics.
Case Example Results.
Relationship of Measures.
Using Internal or External Productivity Benchmarks.
V. PROBLEMS WITH MEASUREMENT PROGRAMS AND HOW TO AVOID THEM.18. Avoiding Obstacles and Common Pitfalls in the Building of an Effective Metrics Program.
Phase I. Requirements Identification and Analysis.
Phase II. Design.
Phase III. Design Implementation.
Phase IV. Program Testing.
Phase V. Implementation and Maintenance.
Biography.19. Unreported and Unpaid Overtime: Distorted Measurements and Formulas for Failure.Gene Fellner
The History of Overtime.
Computerization and Overtime.
The Reasons for Unreported and/or Unpaid Overtime.
A Software Metrics Perspective.
The Impact on Organizations.
The Social Costs.
The Software Engineering Institute's Capability Maturity Model.
A Status Report.
VI. Using Software Metrics for Effective Estimating 32120. Metrics in Support of Estimating in a Large Software Services Company.
The Role of Estimating in Metrics and Process.
Establishing Estimating Centers.
Corporate Metrics for Estimating.
The Business Value of Estimating Centers.
The Case for Model-Based Estimating.
Schedule Compression in Estimating: A Case Study.
Biographies.21. Estimating Software Development Projects.Craig P. Beyers
Why Estimating is Important.
Using Data for Estimating.
How to Estimate Software Projects.
Estimating New Projects or Major Enhancements.
Reestimating Ongoing Projects.
Biography.22. Enhanced Estimation: On Time, Within Budget.David Garmus
Conducting a Proof of Concept.
Selecting a Tool.
Ability To Do What-if Analysis.
Comparisons to Industry and Best-in-Class.
Exporting to Project Management Tools.
Identification of Process Strengths and Weaknesses.
Interface with Function Point Repository.
Portfolio or Baseline Creation (Storage of Data).
Where Do We Go from Here?
Biography.23. Enhanced Estimation.John A. Landmesser
Function Point Counting and Effort Estimation.
Measurement Areas to Evaluate the Estimation Process.
Requirements Analysis Effort Estimation.
Implementation Effort Estimation.
Translating Effort into Client-Friendly Numbers.
Biography.24. ESE: Enhanced Software Estimation.Luca Santillo
The Basic Model: IFPUG Guidelines and ISBSG Benchmark.
To Adjust or Not to Adjust: The VAF Issue.
Internal Benchmarking: The “Best of” Issue.
An Algorithmic Model: Constructive Cost Model.
Function Points and Lines of Code: The Backfiring Issue.
Parametric Models: The Simplicity versus Differentiation Issue.
The Software Reuse Issue.
The Intrinsic Complexity Issue.
Requirements Volatility: The Change Requests Issue.
How Sure Are We: The Uncertainty Issue.
Enhanced Software Estimation: An Integration.
VII. REQUIREMENTS MANAGEMENT.25. The New and Improved Requirements Engineer.
The Importance of Good Requirements.
The Importance of Accurate Estimates.
Why a Software Estimation Group or Consultant is Not Enough.
Other Benefits When Requirements Engineers Have Software Size Expertise.
Function Point Analysis Can Help Determine the Completeness of a Requirements Specification.
Function Point Analysis Becomes More Mainstream.
Biography.26. Early and Quick Function Point Analysis from Summary User Requirements to Project Management.Roberto Meli
Managing the Requirements of a Software Project.
Definition of Requirement.
Requirements Management Framework.
Using Requirements to Estimate Project Resources.
General Software Measurement Framework.
General Software Contractual Framework.
Estimating the Resources of a Software Project.
Measuring or Estimating Function Points.
Early & Quick Function Point Analysis.
VIII. IMPACT ON IT/BUSINESS MEASURES.27. Critical Success Factors for Developing and Implementing a Contractual Metrics Program.
Timely and Robust Definition of Requirements.
Example 1: Batch Processing Performance Measure—Purpose.
Example 2: Batch Processing Performance Measure—Operational Definition.
Example 3: Batch Processing Performance Measure—Research.
Additional Considerations for Definition of Requirements.
Training and Education.
Pre- and Post-implementation Support.
Biography.28. Considerations for Getting Maximum Benefit from an Enterprise-Wide Metrics Repository.Dawn Coley
Getting Started with Storing Collected Metric Data.
Consistency versus Customization.
Maturing the Repository with the Organization.
Enterprise-Wide View and Needs.
Everything but the Kitchen Sink.
Biography.29. The Role of Universities in Promoting Software Measures.Ewa Magiera
The Role of Software Measurement.
The Role of Teaching.
IX. IT BALANCED SCORECARD APPROACHES.30. Measurements Necessary to Support an IT Balanced Scorecard: IT Indicators.
Identification and Selection of Good Indicators for IT Function.
The Balanced Scorecard.
The Balanced Scorecard and IT.
Internal Business Process Perspective.
Innovations and Learning Perspective.
Biography.31. Software Balanced Scorecards: The Icing on the Cake.Bill Hufschmidt
What Is a Software Balanced Scorecard?
Overcoming Fear and Objections.
Using a Software Balanced Scorecard.
Preparing the Software Balanced Scorecard.
Other Important Measures.
Biography.32. Keeping a Well-Balanced Scorecard.Sara Kathryn Shackelton
Positioning Your Company for a Strategic Approach.
Laying Out Your Balanced Scorecard.
Step 1: Create the Company's Vision.
Step 2: Lay Out Your Company's Strategy.
Step 3: Find Measures That Will Determine if You Reach Your Goals.
Step 4: Outline the Formulas For Your Measures.
Step 5: Ensure That Your Measures Use Valid Data.
Step 6: Balance Your Measures.
Step 7: Assign an Owner to Each Measure.
Step 8: Implement Your Balanced Scorecard.
Step 9: What to Do After You Achieve Your Results.
Step 10: Keep Your Measures in Step With Your Company's Progress.
One Final Note.
X. METRICS IN OUTSOURCING.33. Introduction to Metrics in Outsourcing.
Defining and Classifying Outsourcing Arrangements.
Steps in Establishing Metrics for Outsourcing.
Metrics for Outsourcing.
Establishing the Baseline.
Benefits of an Outsourcing Baseline.
Metrics and Outsourcing Major Projects.
Future Trends: Price Models and Independent Benchmarking.
Biography.34. Litigation: The Product of Not Practicing Function Point Metrics.Ron J. Salvador
The Proposal: Will You Dance with Me?.
Silver Bullets Only Work on Werewolves.
Determining Fault: “It's the Interface, Stupid.”
The Phases of Project Development.
The Productivity of Individual Team Members.
Someone Please Tell Me What's Going On Here.
After the Love is Gone.
It's a Bug's Life.
The Requirements They Are A-Changin'.
The Costs of Software Metrics: Are They Worth It?
Biography.35. Metrics in Outsourcing.Koni Thompson
A Recipe for Success.
Planning: Analysis and Identification.
Sample Metrics Profile Outline.
Planning: Capability Analysis.
Governance: Contract Conformance.
Governance: True Up.
A Quality Model.
Recently Published Articles.
XI. SEI AND ISO-BASED METRICS.36. Standardizing a SLOC Counting Tool to Support ISO and CMM Requirements.
Designing and Developing the SLOC Counting Tool.
Impact of SLOC Counting Tool.
Impact of SLOC Counting Tool on Our CMM Journey.
Impact of SLOC Counting Tool on ISO 9000-3 Activities.
Biography.37. Measurement and Analysis in Software Process Improvement.Dennis Goldenson, Joe Jarzombek, and Terry Rout
Why Care about Measurement.
The Need for Explicit and Focused Guidance.
The Need for an Early Focus.
Doing Measurement Right.
Standards and Models.
The Measurement Process.
Practical Software and Systems Measurement.
An Overview of CMMI.
Measurement and Analysis in CMMI.
Describing Good Measurement Practice.
Maturing Measurement Capability.
Implementing Good Measurement Practice.
Maturing Analytic Capability.
Establishing an Effective Measurement Program.
With Special Respect to the Technical Staff.
What Successful Measurement Programs Have In Common.
Summary and Conclusions.
Biography.38. What Can Function Point Analysis Do to Support CMM?Li Hongxing
Putting CMM into Practice.
The Role of FPA in CMM Improvement.
A Repeatable Method for Organizations at Lower Levels.
Control the Scope, Productivity, and Function Requirements.
Putting FPA and CMM into Practice in an Organization.
The Need for Detailed Procedures and Guidelines for FPA and CMM.
The Complexity Matrix and Rating Value Matrix of FPA.
Private Backfiring Standards.
FPA Matures with the Maturing Progress of CMM Status.
Accuracy: Popular Misconceptions.
XII. STATISTICAL PROCESS CONTROL.39. Applying Statistical Process Control to Performance Management.
The Performance Management Challenge.
Testing and Production.
Issues with Data Collection.
Control Charts Defined.
Using SPC as a Management Tool.
Control Charts in Testing.
SPC in the Software Development Life Cycle.
Biography.40. Applying Statistical Process Control to Software.Edward F. Weller
Why Should I Use SPC?
Prerequisites to Using SPC.
Basics of SPC.
Upper and Lower Control Limit Basics.
Inspection Data Analysis.
Control Charts and Predicting Quality.
Cause and Effect Relationships.
Cost to Implement and Use.
XIII. METRICS IN E-COMMERCE.41. The Role of Functional Metrics in B2B E-Commerce Project Success.
B2B E-Commerce Primer.
Overall Function Point Counting and Estimation Methodology.
Benefits of Function Point Counting and Estimation.
Extensions to IFPUG CPM 4.1.
The Function Point Counting and Estimation Repository.
The Function Point Counting and Estimation Team-based Process.
Function Point Estimation Worksheet Example Segment.
Pre-Sales: The Make versus Buy Decision.
Post-Sales: Project Staffing.
During Implementation: Project Control.
Internal Challenges at eSell.
Summary and Conclusions.
Final Questions to Ponder.
Biography.42. Measuring the Usability of E-Commerce Applications.Loredana Frallicciardi
A Usability Measurement Model.
Measuring Usability Subcharacteristics.
Ease of Use.
Biography.43. Metrics in E-Commerce: Function Point Analysis and Component-Based Software Measurements.Jacqueline Jones
What Is Component-Based Software?
What Should We Measure?
Will These Measures Work?
This is a book whose time has come. As the software industry continues to mature, there is an increased emphasis on the need to better understand the impact of technology, methodology, and effective use of skilled resources. Software measurement is one of the key components necessary to effectively monitor progress. Software measurement, for the purposes of this book, is defined as the definition, collection, consolidation, analysis, and reporting of quantitative and qualitative measures within the software development and maintenance arena. Software metrics provide insights into productivity and quality gains realized as a result of applying a combination of skills, technology, and methodology to the software problem domain.
As you begin to read and use this book, you will find a compilation of articles dealing with various aspects of software measurement. The authors include noted industry gurus on the subject, such as Capers Jones and Howard Rubin, as well as writings from both practitioners and service providers. The articles are designed to reflect real-life experiences from which we can all learn.
The contents of this book are intended to serve as a series of reference points on a variety of measurement-related topics. It is unlikely that anyone will pick up the book and read it cover to cover. The audience, therefore, will be as broad as the scope of topics covered. Several of the articles deal with the fundamentals of measurement, while others are for the more advanced or sophisticated practitioner.
The development of this book was part of a volunteer committee effort under the banner of the International Function Point Users Group (IFPUG). However, this compilation of articles is not solely focused on advancing awareness of the use of function points. The book's primary purpose is to provide the audience with the best information available regarding the use of software metrics.
Each of the thirteen parts contains two or more chapters, which are presented alphabetically by author. We have provided a brief biography for each author that will enable you to contact the author directly if you wish to discuss a given topic in greater detail. We also invited the inclusion of a company profile if an author so desired. This form of "advertising" is the only compensation these authors will receive for this book. They contributed their articles with the complete understanding that this was a book intended for the betterment of the software industry as a whole.
The idea for this book came out of the Management Reporting Committee (MRC), one of IFPUG's subcommittees. Its charter it is to support the IFPUG organization with regard to the development and publication of documents necessary to advance the use of software metrics. Once the book was approved by the Executive Board, MRC set about its task of soliciting articles.
A three-tier list of potential authors was compiled that included invitations to industry notables, known practitioners in the field of software metrics, and various membership audiences from a variety of software metrics-related organizations. A range of topics was developed, and responses from those solicited were reviewed based on a common set of acceptance criteria. Once in place, the articles were grouped into sections.
The MRC reviewed all the articles and wrote introductions to the sections. They served as the "agent" between the publisher and the authors, taking on the task of ensuring all reviews and edits were done properly. This book is the result of their tireless efforts, and a note of thanks is appropriate to the following committee members:
David Herron, Committee Chair
Janet Russac, Vice Chair
We should also note the efforts of the IFPUG organization as a whole: the insight of the Board, who authorized the funding for this project, and select members who contributed some of the writings in this book.
The International Function Point Users' Group (IFPUG) is a membership-governed, nonprofit organization committed to increasing the effectiveness of its members' information technology environments through the application of function point analysis (FPA) and other software measurement techniques. IFPUG endorses FPA as its standard methodology for software sizing. In support of this, IFPUG maintains the Function Point Counting Practices Manual, the recognized industry standard for FPA. IFPUG also provides a forum for networking and information exchange that promotes and encourages the use of software product and process metrics. IFPUG members benefit from a variety of services, including
Organized in 1986, IFPUG is one of the fastest growing software metrics user groups in the world. With the addition of its affiliate member organizations, IFPUG serves over 1200 members in more than 30 countries. IFPUG members represent every major industry segment, including aerospace, automotive, banking, insurance, manufacturing, retail, and telecommunications. For further information on the International Function Point User Group, please contact their offices in Princeton Junction, New Jersey (phone number 609-799-4900) or access them electronically on their Web site at www.ifpug.org.
Click below to download the Index file related to this title: