Build in the Quality—Right from the Start!
Traditionally, software developers have not concerned themselves with quality issues until products were at the end of their coding cycles. Now, by applying Dr. W. Edwards Deming's chain approach, developers can plan for quality from the very beginning and implement it throughout the entire course of development.
The methods presented in Inroads to Software Quality will put you on the road to quality certification under:
The Product Delivery Process that lies at the heart of Inroads to Software Quality is self-correcting, market-driven, and customer-oriented. By focusing on Behavioral Testing—in place of traditional functional testing—you can make ongoing evaluation part of the cycle of continuous quality.
Inroads to Software Quality presents approximately 18 quality filters that provide timely and efficient feedback during all phases of development. You will learn how to adapt these filters to your own organization and products with a minimum of waste and rework. The flexibility of this approach allows it to be phased in gradually, without immediately dismantling your organization's existing structure.
The key themes of Process Assurance and Product Assurance underlie every aspect of the book. Individual sections offer thorough coverage of test planning and cycles; outsourcing; quality assurance standards, metrics, and reviews; and the standards and evaluation methods of the most popular quality assurance certifications.
The book's Process Improvement Roadmap explains the seven crucial steps for process improvement and presents a detailed plan for implementing them. Dozens of checklists and templates help you put the material to work right away. The accompanying diskette provides these checklists and a description of the major filters. It includes the configuration board impact assessment filter, criteria for code audits, the code audit filter, and much more!
Best of all, the authors present their concepts in a simple, user-friendly style. No statistics, no equations, just practical step-by-step explanations to help you make software development both more efficient and more responsive to user needs.
1. Process Assurance.
Techniques for Process Assurance.
Project Team. Project Standards. Schedule Monitoring. Project Tracking. Estimation. Effective Communication. Steering Committee. Project Risks. Measurement. Integrated Technology (IT). Final Comments.
Causes of Failure in Process Assurance.
Lack of Management Support. Lack of User Involvement. Lack of Project Leadership. Lack of Measures of Success.
Verification versus Validation.
Verification and Validation Plan (VVP). Who Conducts the V&V?
The Product Delivery Process (A New Paradigm).
Basic Concepts. Product Delivery Process. The Importance of the Project Plan. Configuration Control Board. The Efficiency of a Software Development Strategy.
Techniques for Product Assurance.
Establishing Standards for Product Quality. Establishing Standards for Development Technology. Joint Application Development. Prototyping. Disaster Recovery. Configuration Management. Reusability. Code Management. Software Changes. Training. Management Support. Benchmarking.
Activities to Assure Product Quality. A New Paradigm for Assuring Quality Software Design and Architecture.
Dependency Analysis at the Module Level. Eliminating Functional Testing at the Module Level. Dependency Analysis at the System Level. Eliminating Common System-Level Dependencies. Examples of the Basic Tools in Dependency Analysis at the System Level. Final Comments.
Required Components of Software Quality. Building an Effective Software Quality Assurance Organization.
Quality Assurance. Verification and Validation. Simulation. Testing.
Building a Separate Quality Assurance Organization. The Cost-Effectiveness of Software Testing.
Credibility and Return on Investment. Different Projects Require Different Approaches. Begin Collecting Data. Making a Case for Finding Errors Early in the Life Cycle. Steps to Creating an Effective Test Organization. How to Create an Effective, Stable Test Organization. Making a Test Organization More Effective. Conclusion.
The Impact of the New Paradigm.
Management Aspects (Return on Investment) under the New Paradigm. Leverage at the Software Product Level. Guaranteeing Leverage.
Techniques and Contents of a SQA Plan.
What to Address in Your SQA Plan.
Some Standards for Software Quality.
The Concept of Six Sigma. Reducing Cycle Times. Continuous Process Improvement.
The Product Delivery Process.
Deliverables Associated with a Product Delivery Process. A Description of the Marketing Requirements Definition.
Components of a Specification.
Functional Specification. Information Model. Constraint Document. Performance Criteria Document. Hardware Specification. Preliminary Budget/Preliminary Schedule/Preliminary Manpower Allocation. Standards Document. Quality Plan. Master Test Plan. Documentation Plan. Product Release Plan. A Test Plan for System Testing.
Release Management. Software Quality Standards in the Product Delivery Process.
What Is Software Quality? An Alternative Approach to Quality Measurement. What We Mean by Software Quality. Software Quality Standards.
Objectives of Testing.
Error Detection. Error Removal. Error Tracking. Regression Testing. Goals of Software Testing. Error-Prone Code and Failure Modes. Effective Test Team Validation Strategies.
Black Box Testing versus White Box Testing. Unit Testing.
Ad Hoc Testing. White Box Testing. Black Box Testing.
Incremental versus Nonincremental. Big-Bang Integration. Top-Down Integration. Bottom-Up Integration. Risk-Based Integration Testing. Threaded Integration Testing. Outside-In Integration Testing (Essential Systems Analysis). Inside-Out Integration Testing (Orr DSSD-Data Structured Systems Development). Little-Bang Integration Testing. The New Paradigm (The Product Delivery Process).
System Test Considerations. Fundamental Tests (Product Verification Testing). Other Tests (Measures). Testing/Maintenance Problems. The Credibility of Software Testing. A Comment on Beta Testing. A Comment on Regression Testing. Acceptance Tests.
Benefits of a Test Plan.
Techniques. Steps to Perform.
Developing the Successful Test Plan. Test Plan Sample. Creating Test Cases.
Techniques and Characteristics.
Automated Test Tools.
Evaluation Criteria. Test Results. Test Case Generation. Vendor Selection. The Pilot Project Approach.
Background. Potential Problems. Contract Negotiations.
Business Review. Technical Review. Management Review. Roles and Responsibilities.
Review Frequency. Review Planning. Review Meeting. Rework. Follow-Up.
Walk-throughs and Inspections.
Definitions. Understanding the Need for Collecting Metrics. Benefits of Metrics. Cost of Metrics. Common Measurements and Tips.
Requirements. Code/Design. Testing.
Seven Commonly Tracked Measures.
Number of Defects. Work Effort. Schedule. Number of Changes to the Requirements. Size. Documentation Defects. Complexity.
Establishing a Measurement Program and Prototyping.
Prototyping. Measurement Cycle.
The Impact of Filters as Feedback Mechanisms. The Misuse of Statistics in Software Development.
Statistical Measurements. Sampling versus Case Studies. Errors of the Estimate in Statistical Inference. Statistics as Applied to Quality Control. Conclusion.
Seven Steps of the Process Improvement Road Map.
Plan. Gather Data. Analyze Findings. Describe the Ideal Process. Step Implement the Ideal Process. Measure Progress. Standardize the Process.
SEI Maturity Model. Relationship and Differences Between SEI, ISO 9001, and ISO 9001-3. Conclusion.
The Umbrella of Total Quality Management. TQM and the New Paradigm. Risk Issues in Software Quality Assurance.
Areas of Risk. Decision Making Based on Risk. Minimizing the Impact of Risk. Minimizing Risk through Increasing Testability. Building a First-Rate Testing Organization.
Summary and Conclusions.
Test Case Form. Checklist for a Vendor-Developed Package. Development Life Cycle Checklist. Conceptual Design Checklist. Detail Design Checklist. Systems Development Checklist. Training and Implementation Checklist. Support and Operations Checklist. Testing Checklist. Documentation Checklist. System Maintenance Checklist. Vendor Selection Checklist. Review Checklist. Product Assurance Checklist. Test Plan Template. Where to Obtain Guidelines/Standards. Descriptions of Frontend Filters for the Product Delivery Process. Description of Process for Configuration Control Board Impact Assessment Document Changes to Frozen Applications. Strategy for Minimizing the Number of Correct Solutions in a Module. Criteria for Code Audits. Thoughts on the Theory Behind the New Product Development Process and a Proposed Code Audit Filter. Thoughts on the Difference Between Advertising, Sales, and Marketing.
Welcome to Inroads to Software Quality.
As we compiled this book, we recognized that there are several traditions associated with software quality.
The old-fashioned methods of building software, with various attempts to "fix things up" after the fact.
The new quality-oriented methods, as characterized by total quality management, ISO 9000, SEI CMM levels, and the Malcolm Baldrige National Quality Awards, among others.
A new shift of paradigm which radically changes how software is built and delivered, which eliminates the need to perform traditional functional testing, in terms of unit testing, integration- and system-level functional testing, and regression testing. This is based on a well-defined, disciplined quality process. As you might expect, it is a controversial approach. It allows more time for performing behavioral testing. Normally, testing experts refer to functional testing as being the same thing as black box testing, and often, these are referred to as behavioral testing. We will use behavioral testing to relate to how a software system performs, rather than what it does.
The product delivery process which implements this paradigm is based on market-driven product development, using filters to build quality into deliverables, and designing quality into software products at the outset. The process is based on constant feedback and continuous process improvement.
Most companies are attempting to move from the first approach to the second. Almost all quality initiatives are based on the second approach, so we will present that approach as the basic method for improving the quality of products. In addition, we will present a preliminary view of the new paradigm and show why it works and what a tremendous positive impact it can have on software quality-and on cost and schedule. Although software development and testing concepts can quickly become very complex, we attempt to keep this book as an easy-to-follow training guide that offers traditional quality assurance activities and core elements of testing required to improve the quality of the products.
In addition, we will provide new definitions of software quality and show techniques for achieving a higher level of quality than has been previously possible for most companies. In order to keep these techniques simple, some of them may appear dated; others may have different names and definitions from the standard. We will try to state the pro's and con's of each of these and provide ref- erences to sources of the various approaches.
A lot of what we present will be in the spirit of the quality concepts of W. Edwards Deming as described by Hahn (1995). You may be surprised to find that when these concepts are actually laid alongside the traditional software quality practices, they become very controversial. They have a major impact on the con- cepts of inspections, application of statistical quality control, inferential statis- tics as opposed to analytical statistics (case studies), software development strategies, life cycle models, software delivery systems, and testing.
There is a temptation to take each of the traditional and new testing tech- niques and critique them in terms of our new paradigm, but that would go beyond the purpose of this book. This book is mainly about quality and quality processes and not about testing. Many of the testing concepts summarized in this book are considered to be dated. Newer concepts, such as those found in Beizer (1991, 1995), come from the current refereed literature. Unfortunately, they are also based on the fact that the code must be taken as it has been programmed and tested in that state.
Our approach shows that much of what the new test techniques test for can be designed out. Many of the new techniques do not test against interface stan- dards such as Microsoft® Windows®, etc. Nor do they test for problems related to event-driven environments. With good design, it is possible to make an entire software system deterministic rather than stochastic. This makes it possible to predict and create whatever condition you want to examine.
Because of the extreme importance of management commitment and disci- pline within the framework of the new paradigm, the book will outline the importance of managing software development and provide a strategy for doing so. Apart from this, there will be no general discussions of personnel manage- ment, statistical quality control, or the concepts of code complexity or estimation. We will present a software development strategy which provides the basis for the new software development paradigm.
The book will present information in layers. Rather than trying to cover all aspects of a topic in one place, the book presents information in small, nonintim- idating chapters. In addition, there is a chapter on the relationship among total quality management and our concepts of software quality, managing risks, and the use of the new paradigm to increase quality. The most important aspect of this book is that each chapter contains detailed instructions on how to conduct some of the activities discussed and each chapter can be read independently. This calls for some duplication of terms and ideas throughout the book.
An introductory chapter on the techniques for process assurance is followed by a chapter on product assurance, techniques for product assurance, and plan- ning and organization. Chapters 3 through 6 cover software quality assurance, software quality standards, an overview of test cycles, and test planning. Chap- ter 6 gives details on how to develop a test plan and test cases. Samples, outlines, and templates for a test plan and a test case are included. The two chapters that follow discuss software quality assurance reviews and basic concepts of software measurement. Chapter 7 outlines objectives, techniques, and guidelines for con- ducting successful quality assurance reviews. Chapter 8 focuses on how to start a measurement program, important issues, and some of the most commonly used engineering measures in the software. Chapter 9 presents a seven-step process improvement road map, with details on activities to be performed and other con- siderations. Chapter 10 discusses standards and evaluation of the software development process in the context of ISO 9000-3 and SEI maturity model. The final chapter addresses the process of achieving total quality management and how to minimize the risks involved in developing quality products.
The Appendix provides various checklists that can be used in their present format or customized, depending on your needs and the size of your project. If these checklists are enhanced by defining the "quality" and "accuracy" required for each item, then these checklists become filters. The Appendix also includes a description of some front-end filters and an example of the Configuration Control Board Impact Assessment document, which can be extremely useful in helping manage change.
The accompanying DOS-format diskette duplicates the content of the Appendix for use on your Mac or PC in three word-processing programs: WordPerfect® 5.1 for Windows, Microsoft Word® 5.1 for Macintosh®, and Microsoft Word 2.0 for Windows. The files are separated in two portions: "Templates/Checklists" and "Descriptions of Front-End Filters for Product Deliv- ery Process." The WordPerfect files have the extension .wp, the Microsoft Word for Windows files have the extension .msw, and the Word for Mac files have the extension .mac.
Inroads to Software Quality is based on our experience over many years and the input of a large number of individuals and organizations. We therefore have not attempted to tie all the concepts back to a particular person or refer- ence. Where we think you might benefit from additional reading, we have pro- vided references.
In summary, this book is intended to appeal to a broad range of readers, including:
individuals with little computer background wanting to begin a quality assurance program
those with testing backgrounds who want advice as to how to implement an effective testing program
testing personnel who wish a quick overview of the testing field
programmers who want to improve the quality of their code
software developers and architects who want to improve the quality of their software architectures and designs
software managers who wish to install a measurement program to help them evaluate their quality processes
high-level managers, middle-level managers, and software managers who wish to implement an effective quality program wherein quality is defined from a marketability point of view and is designed in at the start
There should be no need for prerequisites to be able to read this book. But the reader should have an orientation or context based on software quality. Even in Chapter 2, where simple program structures are presented, the reader should be able to relate these to any process: cooking recipes, instructions for operating or building some device, directions for how to get from one place to another, etc.
A Software Glossary with Commentary is at the end of the book to help you understand the material better and to provide quick summaries of specific topics.
Many people contributed to the ideas in this book. Others helped us evaluate the accuracy and readability of the various parts of the book. Still others read and disagreed with us, especially in the more "novel" concepts of the new paradigm, helping us make more precise the arguments and examples we have presented.
We would especially like to thank Digital Technology International, where our product development process and the front-end filters were implemented and validated. We would not have been able to validate the success of the new para- digm without their cooperation and support.
Our special thanks go to Linda Hayes for her input to the chapter on auto- mated testing and to Nate Oldham for drawing most of the diagrams in Chapter 2.
We would also like to thank Karen Snow, Estella Weems, Cindy Snow of Intel, Shauna Frandsen and Tracy Weymouth of Digital Technology Interna- tional, and Amy Buckwalter of Utah County Journal. Thanks also go to Connie Marchewka of Texas Instruments for providing us with information on TI's award-winning quality programs, Nancy Cadjun for technical writing, and Cem Kaner for reviewing and giving feedback on vendor-supplier legal issues.
Finally, we wish to acknowledge our publisher, Bernard Goodwin, and our production editor, Jane Bonnell, and to thank them for their patience and guidance.