Register your product to gain access to bonus material or receive a coupon.
Testing Embedded Software
- By Bart Broekman, Edwin Notenboom
- Published Dec 10, 2002 by Addison-Wesley Professional.
Book
- Sorry, this book is no longer in print.
About
Features
- Provides a practical and comprehensive overview of how to test embedded software.
- Includes classic as well as modern test design techniques.
- Demonstrates an approach that is useful in the real-life situations of 'limited time and resources.'
- In depth treatment of test design techniques that can be directly applied.
- Description of how to establish a professional test organision - with particular emphasis on roles, tasks, responsibilities, organisational hierarchies and communication structures.
- Coverage of how to organise the test process.
Description
- Copyright 2003
- Dimensions: 7-3/8x9-1/4
- Pages: 368
- Edition: 1st
- Book
- ISBN-10: 0-321-15986-1
- ISBN-13: 978-0-321-15986-1
The book provides a practical and comprehensive overview of how to test embedded software.
- The book describes how embedded systems can be tested in a structured, controlled way.
- The first complete description of all necessary ingredients of a testing process.
- It includes classic as well as modern test design techniques.
- The described approach is useful in real-life situations of 'limited time and resources.
More and more our society is pervaded by embedded software: cars, telecom, home entertainment devices are full of software. Embedded systems are becoming larger and more complex with an increasing amount of software, leading to a growing need for a structured testing method which helps to tackle the typical problems in embedded software testing.
Audience:
Managers or team leaders that are responsible for development and/or testing of embedded software and systems. Also, people who actually perform the primary software testing activities.
User level:
Intermediate.
Bart Broekman has been a software test practitioner since 1990. He participated in European embedded software research projects (ITEA) and is co-author of a book on test automation.
Edwin Notenboom has been a professional tester at Sogeti for six years. Together with Bart Broekman, he participated in a european ITEA project on embedded systems since February 1999.
Sample Content
Table of Contents
Foreword
Preface
Acknowledgments
Recommendations
PART I INTRODUCTION
1. FUNDAMENTALS
1.1 Aims Of Testing
1.2 What Is An Embedded System?
1.3 Approach To The Testing Of Embedded Systems
2. THE TEMB METHOD
2.1 Overview
2.2 Temb Generic
2.3 Mechanism For Assembling The Dedicated Test Approach
PART II LIFECYCLE
3. MULTIPLE V-MODEL
3.1 Introduction
3.2 Test Activities In The Multiple Vs
3.3 The Nested Multiple V-Model
4. MASTER TEST PLANNING
4.1 Elements Of Master Test Planning
4.2 Activities
5. TESTING BY DEVELOPERS
5.1 Introduction
5.2 Integration Approach
5.3 Lifecycle
6. TESTING BY AN INDEPENDENT TEST TEAM
6.1 Introduction
6.2 Planning And Control Phase
6.3 Preparation Phase
6.4 Specification Phase
6.5 Execution Phase
6.6 Completion Phase
PART III TECHNIQUES
7. RISK-BASED TEST STRATEGY
7.1 Introduction
7.2 Risk Assessment
7.3 Strategy In Master Test Planning
7.4 Strategy For A Test Level
7.5 Strategy Changes During The Test Process
7.6 Strategy For Maintenance Testing
8. TESTABILITY REVIEW
8.1 Introduction
8.2 Procedure
9. INSPECTIONS
9.1 Introduction
9.2 Procedure
10. SAFETY ANALYSIS
10.1 Introduction
10.2 Safety Analysis Techniques
10.3 Safety Analysis Lifecycle
11. TEST DESIGN TECHNIQUES
11.1 Overview
11.2 State Transition Testing
11.3 Control Flow Test
11.4 Elementary Comparison Test
11.5 Classification-Tree Method
11.6 Evolutionary Algorithms
11.7 Statistical Usage Testing
11.8 Rare Event Testing
11.9 Mutation Analysis
12. CHECKLISTS
12.1 Introduction
12.2 Checklists For Quality Characteristics
12.3 General Checklist For High-Level Testing
12.4 General Checklist For Low-Level Testing
12.5 Test Design Techniques Checklist
12.6 Checklists Concerning The Test Process
PART IV INFRASTRUCTURE
13. EMBEDDED SOFTWARE TEST ENVIRONMENTS
13.1 Introduction
13.2 First Stage: Simulation
13.3 Second Stage: Prototyping
13.4 Third Stage: Pre-Production
13.5 Post-Development Stage
14. TOOLS
14.1 Introduction
14.2 Categorization Of Test Tools
15. TEST AUTOMATION
15.1 Introduction
15.2 The Technique Of Test Automation
15.3 Implementing Test Automation
16. MIXED SIGNALS
16.1 Introduction
16.2 Stimuli Description Techniques
16.3 Measurement And Analysis Techniques
PART V ORGANIZATION
17. TEST ROLES
17.1 General Skills
17.2 Specific Test Roles
18. HUMAN RESOURCE MANAGEMENT
18.1 Staff
18.2 Training
18.3 Career Perspectives
19. ORGANIZATION STRUCTURE
19.1 Test Organization
19.2 Communication Structures
20. TEST CONTROL
20.1 Control Of The Test Process
20.2 Control Of The Test Infrastructure
20.3 Control Of The Test Deliverables
PART VI APPENDICES
Appendix A: Risk Classification
Appendix B: Statecharts
B.1 States
B.2 Events
B.3 Transitions
B.4 Actions And Activities
B.5 Execution Order
B.6 Nested States
Appendix C: Blueprint Of An Automated Test Suite
C.1 Test Data
C.2 Start
C.3 Planner
C.4 Reader
C.5 Translator
C.6 Test Actions
C.7 Initialization
C.8 Synchronization
C.9 Error Recovery
C.10 Reporting
C.11 Checking
C.12 Framework
C.13 Communication
Appendix D: Pseudocode Evolutionary Algorithms
D.1 Main Process
D.2 Selection
D.3 Recombination
D.4 Mutation
D.5 Insertion
Appendix E: Example Test Plane.1 Assignment
E.2 Test Basis
E.3 Test Strategy
E.4 Planning
E.5 Threats, Risks, And Measures
E.6 Infrastructure
E.7 Test Organization
E.8 Test Deliverables
E.9 Configuration Management
Glossary
References
Preface
A growing industry requires structured testing
The embedded systems world is a fast growing industry. It is a world which is historically dominated by engineers and technicians who excel in their own technical specialism. Historically, the technicians who built the products were also those who performed the testing because they understood best how things were supposed to work. This worked fine in orderly situations where the technicians worked on relatively small and isolated products. However, the embedded industry is changing fast — systems have become larger, more complex, and more integrated. Software now makes up the larger part of the system, often replacing hardware. Systems that used to work in isolation are linked to provide integrated functionality. This cannot be produced by one brilliant individual anymore, but has to be an organized team effort. Similarly, the process of testing has become larger, more complex, and harder to control. It has led to a growing need for a method that helps get the complex testing process under control.
Scope of this book
Embedded systems have to rely on high quality hardware as well as high quality software. Therefore, both hardware testing and software testing are essential parts of the test approach for an embedded system. However, this book concentrates more on the testing of
software in embedded systems. Many hardware issues are included, but technical details of testing individual hardware components are not discussed — this is a profession in itself. Usually the technical people are competent in dealing with the technical intricacies involved in the testing of hardware on a detailed level. This book is mainly targeted at those who work with the software in embedded systems. It teaches them about the environment in which they work, the specific problems of testing their software, and techniques that are not normally taught in software education.This book aims at providing answers and solutions to the growing problem of "getting the complex testing process under control." It focuses on the higher level of
how to organize the overall testing process with its broad range of activities in both software and hardware environments. The authors have used concepts and material from the book Software Testing, a Guide to The TMap® Approach, and have adapted them to fit the embedded software world.The book is not intended to have the academic quality of a thesis. It is strongly practically oriented and aims at providing overview, insight, and lots of practical guidelines, rather than detailed academic proof.
Structure of the book
Testing is more than just exercising the system and checking if it behaves correctly. It also involves the planning of activities, designing test cases, managing the test infrastructure, setting up an organization, dealing with politics and much more. This book describes the TEmb method for structured testing of embedded software. It covers the broad range of issues that is essential in structured testing, answering the questions "what, when, how, by what and by whom?" TEmb uses the four cornerstones of structured testing as defined by the test management approach TMap® (Pol
et al., 2002): lifecycle ("what, when") of the development and testing process; techniques ("how"); infrastructure ("by what"); organization ("by whom"). The structure of this book follows those four cornerstones.The book is divided into six parts:
Part I
describes some general principles of structured testing and embedded systems. It provides an overview of the TEmb method, showing how to assemble the suitable test approach for a particular embedded system.Part II deals with lifecycle issues and thus with the process of developing and testing embedded software. The lifecycle cornerstone is the core of the testing process, providing the map of what has to be done and in what order. Various issues from the other three cornerstones apply at various points in that lifecycle.
Part III offers several techniques that are probably useful for most embedded software test projects. It contains techniques for executing a risk-based test strategy, a testability review, formal inspections, and safety analysis. It offers various techniques for designing test cases that can be used in different circumstances for different purposes.
Part IV deals with the infrastructure that testers need to do their job properly. It describes the different test environments required at different stages in the testing process. An overview is provided of the various tools that can be usefully applied for different test activities and purposes. Tools that assist in achieving automated test execution have always been very popular. This part explains the technical and organizational issues involved in this kind of test automation. Finally, specific issues are discussed that arise when the tester works in an environment dealing with analog as well as digital signals (so-called "mixed signals").
Part V describes various organizational aspects of testing. It is about those who have to perform the testing activities and who have to communicate with other personnel. It contains descriptions of the various test roles as well as the management and organizational structures. It also deals with how the testers should report on the progress of testing and on the quality of the system under test.
Part VI contains various appendices with background information on subjects such as risk classification, statechart models, a blueprint of an automated test suite and an example test plan.
Target audience
This book is targeted at those who are involved in the development and testing of embedded systems (and embedded software in particular). It offers guidelines on organizational as well as technical aspects, on a global as well as a detailed level. Different types of readers will probably have different "favourite chapters." The following may serve as a guideline as to which chapters will be most relevant for you.
It is recommended that all read the introductory chapters in Part I as well as the chapters on the multiple V-model, master test planning, and risk-based test strategy. They encapsulate the essence of the TEmb method.
Managers of development or test projects, test co-ordinators or test team leaders will benefit most from the chapters in Part II and Part V plus the chapter on "risk-based test strategy."
Testers, developers, and others who actually perform the primary software testing activities will find a lot of practical information in Part III and Part IV. If the reader is required to report formally on progress and quality, they will benefit from reading the chapter on test control in Part V.
Those who are involved in development or testing of hardware are advised to read the chapters on the multiple V-model in Part II and embedded software test environments in Part IV. They show that it is essential for both disciplines (hardware and software) to work towards a common goal and synchronize efforts to reach this goal. For the same reason, those chapters are of interest to software developers and testers.
Human resource management departments will find information that is especially suited to their line of work in the chapters on test roles and human resource management in Part V.
Acknowledgments
It will be difficult to avoid clichés such as "This book would never ....," but we just don’t want to ignore the fact that we received valuable help from many people. Mentioning their contribution here is the least they deserve.
We got the chance to develop a testing method for the embedded industry when Rob Dekker and Martin Pol asked us (very persuasively) to join a European ITEA project on development methods for embedded systems in the automotive industry. Klaas Brongers managed and stimulated our efforts, providing the required freedom and creative atmosphere. The project proved to be a fertile soil for developing a test method for embedded software, where we could research new ideas and test their practical value. Especially the project partners Mirko Conrad, Heiko Dörr and Eric Sax were an invaluable source of knowledge and experience. They were very supportive as well as highly critical, which greatly increased the quality of our work. We owe a special thanks to Mirko and Eric who provided a complete chapter about the specialized subject of mixed signals.
Somewhere during the ITEA project, it became apparent that the ideas developed in the project would be useful for so many others in the embedded industry — so the idea of the book was born. We were fortunate to have managers and directors with the vision and determination required to pursue such a goal: Luciëlle de Bakker, Hugo Herman de Groot, Jan van Holten, Ronald Spaans, and Wim van Uden. They provided the essential management support.
Once the decision was made to write a book, we knew that we could count on the support of many colleagues. Bart Douven, John Knappers, Peter van Lint, Dre Robben, Coen de Vries, and Paul Willaert provided us with valuable input and feedback. Peter and Coen should be mentioned in particular. They put a lot of effort into developing the thoughts about embedded software test environments that formed the basis of the chapter about this subject. Rob Baarda was a great help in the process of turning a document into a book and getting it published. Jack van de Corput and Boy van den Dungen succeeded in raising enthusiasm at customer sites to put our ideas into practice.
As more and more chapters were completed, external reviews were intensified. We are very happy that we could take advantage of experts in the fields of testing and embedded software: Simon Burton, Rix Groenboom, Klaus Grimm, Norbert Magnusson, Matthias Pillin, and, in particular, Stuart Reid and Otto Vinter who reviewed the complete final manuscript. We appreciate that they didn’t spare us, but gave us their honest opinions and constructive criticism. Many of their remarks and suggestions have been accepted and found a place in this book.
We want to thank all those people for the invaluable contributions to our book. They can all feel as proud of this work as we do.
Bart Broekman
Edwin Notenboom
10321159861P10152002
Foreword
The importance of software is increasing dramatically in nearly all sectors of industry. In the area of business administration, software has become an indispensable core technology. Furthermore, more and more products and innovations are also based on software in most technical fields. A typical example is provided by the automotive industry in which a rapidly increasing number of innovations are based on electronics and software to enhance the safety of the vehicles, and also to improve the comfort of the passengers and to reduce fuel consumption and emissions. In modern upper-class and luxury cars, 20 to 25 percent of the cost is on electronics and software, and this proportion is estimated to increase to up to 40 percent in the next ten years.
Software has a substantial influence on the quality of products as well as the productivity of a company. Practice, unfortunately, shows that it is impossible to develop a complex software-based system "first-time-right". Hence, comprehensive analytical measures have to be taken to check the results of the different development phases and to detect errors as early as possible. Testing constitutes the most important analysis technique, besides reviews and inspections. It is, however, a very sophisticated and time consuming task, particularly in the field of embedded systems. When testing embedded software, not only the software has to be considered but also the close connection to the hardware components, the frequently severe timing constraints and real-time requirements, and other performance-related aspects.
This book should be regarded as an important and substantial contribution to the significant improvement of the situation in the field of testing embedded systems. It provides a comprehensive description of the world of embedded software testing. It covers important aspects such as the testing lifecycle and testing techniques, as well as infrastructure and organization. The authors’ concentration on usability in industrial practice makes the book a valuable guide for any practitioner. Due to the wide range of application of embedded software, the book will be useful in many industrial businesses.
With its comprehensiveness and practical orientation this book provides a significant milestone on the long road to the effective and efficient testing of embedded software, and thus to the economic development of high-quality embedded systems. Several concepts from the book have already established a foothold in the DaimlerChrysler Group and I hope that it will find its way to many testers and software developers in various different industrial sectors in which the development of dependable embedded systems is part of the core business.
Dr. Klaus Grimm, Berlin, June 2002
Director of Software Technology Research at DaimlerChrysler AG