Home > Articles > Process Improvement

Defining Software Quality and Economic Value

This introduction to The Economics of Software Quality covers how to define software quality and economic value.
This chapter is from the book

This chapter is from the book

Introduction

This book deals with two topics that have been ambiguous and difficult to pin down for many years: software quality and economic value.

The reason for the ambiguity, as noted in the Preface, is that there are many different points of view, and each point of view has a different interpretation of the terms. For example, software quality does not mean the same thing to a customer as it does to a developer. Economic value has a different meaning to vendors than it has to consumers. For vendors, revenue is the key element of value, and for consumers, operational factors represent primary value. Both of these are discussed later in the book.

By examining a wide spectrum of views and extracting the essential points from each view, the authors hope that workable definitions can be established that are comparatively unambiguous.

Software quality, as covered in this book, goes well beyond functional quality (the sort of thing to which customers might react to in addition to usability and reliable performance). Quality certainly covers these aspects but extends further to nonfunctional quality (how well the software does what it is meant to do) and to structural quality (how well it can continue to serve business needs as they evolve and change as business conditions do).

Why Is Software Quality Important?

Computer usage in industrial countries starts at or before age 6, and by age 16 almost 60% of young people in the United States have at least a working knowledge of computers and software. Several skilled hackers have been apprehended who were only 16 years of age.

The approximate population of the United States in 2010 was about 309,800,135 based on Census Bureau estimates. Out of the total population about 30% use computers daily either for business purposes or for recreational purposes or both; that is, about 92,940,040 Americans are daily computer users.

About 65% of the U.S. population use embedded software in the form of smart phones, digital cameras, digital watches, automobile brakes and engine controls, home appliances, and entertainment devices. Many people are not aware that embedded software controls such devices, but it does. In other words, about 201,370,087 U.S. citizens own and use devices that contain embedded software.

Almost 100% of the U.S. population has personal data stored in various online databases maintained by the Census Bureau, the Internal Revenue Service, state governments, municipal governments, banks, insurance companies, credit card companies, and credit scoring companies.

Moving on to business, data from various sources such as Forbes, Manta, Business Week, the Department of Commerce Bureau of Labor Statistics, and others reports that the United States has about 22,553,779 companies (as of the end of 2010). Of these companies about 65% use computers and software for business operations, retail sales, accounting, and other purposes—so about 14,659,956 U.S. companies use computers and software. (Corporate software usage ranges from a basic spreadsheet up to entire enterprise resource planning [ERP] packages plus hundreds of other applications.)

Based on data from the Manta website, the software deployed in the United States is provided by about 77,186 software companies and another 10,000 U.S. companies that create devices with embedded software. A great deal of embedded software and the device companies themselves have moved to China, Taiwan, Japan, India, and other offshore countries. An exception to offshore migration is the manufacture of embedded software for military equipment and weapons systems, which tends to stay in the United States for security reasons.

The U.S. military services and the Department of Defense (DoD) own and deploy more software than any other organizations in history. In fact, the DoD probably owns and deploys more software than the military organizations of all other countries combined. Our entire defense community is now dependent on software for command and control, logistics support, and the actual operation of weapons systems. Our national defense systems are highly computerized, so software quality is a critical component of the U.S. defense strategy.

Without even knowing it, we are awash in a sea of software that operates most of our manufacturing equipment, keeps records on virtually all citizens, and operates the majority of our automobiles, home appliances, and entertainment devices. Our transportation systems, medical systems, and government operations all depend on computers and software and hence also depend on high software quality levels.

While software is among the most widely used products in human history, it also has one of the highest failure rates of any product in human history due primarily to poor quality.

Based on observations among the authors' clients plus observations during expert witness assignments, the cancellation rate for applications in the 10,000 function point size range is about 31%. The average cost for these cancelled projects is about $35,000,000. By contrast, projects in the 10,000 function point size range that are successfully completed and have high quality levels only cost about $20,000,000.

When projects developed by outsource vendors are cancelled and clients sue for breach of contract, the average cost of litigation is about $5,000,000 for the plaintiff and $7,000,000 for the defendant. If the defendants lose, then awards for damages can top $25,000,000. However because most U.S. courts bar suits for consequential damages, the actual losses by the defendants can be much larger.

Of the authors' clients who are involved with outsourcing, about 5% of agreements tend to end up in court for breach of contract. The claims by the plaintiffs include outright failure, delivery of inoperable software, or delivery of software with such high defect volumes that usage is harmful rather than useful.

As of 2011, the average cost per function point in the United States is about $1,000 to build software applications and another $1,000 to maintain and support them for five years: $2,000 per function point in total. For projects that use effective combinations of defect prevention and defect removal activities and achieve high quality levels, average development costs are only about $700 per function point and maintenance, and support costs drop to about $500 per function point: $1,200 per function point in total.

Expressed another way, the software engineering population of the United States is currently around 2,400,000 when software engineers and related occupations such as systems analysis are considered. On any given day, due to poor quality control, about 1,000,000 of these workers spend the day finding and fixing bugs (and, unwittingly, injecting new bugs as part of the process).

So all of these statistics point to the fact that better software quality control in the forms of defect prevention and more effective defect removal could free up about 720,000 software personnel for more productive work than just bug repairs, easily reducing U.S. software development and maintenance costs by about 50%.

As we show later in the book, the cost savings that result from higher quality are proportional to application size. As software projects grow larger, cost savings from high quality levels increase. Table 1.1 illustrates typical software development costs for low, average, and high-quality software applications.

Table 1.1. Software Costs by Size and Quality Level

(Burdened cost = $10,000 per month)

Function Points

Low Quality

Average Quality

High Quality

10

$6,875

$6,250

$5,938

100

$88,561

$78,721

$74,785

1,000

$1,039,889

$920,256

$846,636

10,000

$23,925,127

$23,804,458

$18,724,012

100,000

$507,767,782

$433,989,557

$381,910,810

The technologies and methods associated with these three quality levels are discussed and illustrated in later sections of this chapter, as are the reasons that large software projects are so risky. Suffice it to say the "high quality" column includes effective defect prevention, effective pretest defect removal such as inspections and static analysis, and much more effective testing than the other columns.

Another major reason that software quality is important is because poor quality can and will affect each citizen personally in unpleasant ways. Every time there is a billing error, every time taxes are miscalculated, every time credit ratings change for incorrect reasons, poor software quality is part of the problem.

Early in 2010, hundreds of computers were shut down and many businesses including hospitals were disrupted when the MacAfee antivirus application mistakenly identified part of Microsoft Windows as a virus and stopped it from loading.

According to the July 25, 2010, issue of Computerworld, the BP drilling platform that exploded and sank had been having frequent and serious computer problems for a month prior to the final disaster. These problems prevented significant quantities of data from being analyzed that might have warned operators in time to shut down the oil pumping operation.

If your automobile braking system does not operate correctly, if a home appliance fails unexpectedly, or if a hospital makes a medical mistake, there is a good chance that poor software quality was part of the problem.

If an airline flight is delayed more than about two hours or if there is a widespread power outage that affects an entire geographic region such as New England, the odds, again, are good that poor software quality was part of the problem.

Because software is such a basic commodity as of 2011, it is useful to start by considering how much software ordinary U.S. citizens own and use. Table 1.2 shows typical software volumes associated with normal living activities.

Table 1.2. Personal Software Circa 2011

Products

Function Points

Lines of Code

Daily Usage Hours

Personal computer

1,000,000

50,000,000

2.00

Automobile

350,000

17,500,000

2.00

Smart appliances

100,000

5,000,000

1.00

Smart phone

25,000

1,250,000

1.50

Social networks

25,000

1,250,000

1.50

Home entertainment

10,000

500,000

2.00

Electronic book

5,000

250,000

1.00

Digital camera

2,500

125,000

0.50

Digital watch

1,500

75,000

0.50

TOTALS

1,519,000

75,950,000

12.00

The data in Table 1.2 comes from a combination of web sources and proprietary data provided by clients who build appliances of various kinds.

Not every citizen has all of these appliances and devices, but about half of us do. Many of us have even more than what Table 1.2 indicates, such as owning several automobiles, several cell phones, and numerous appliances. Software quality is important because it is the main operating component of almost all complex machines as of 2011.

Another reason that software quality is important is because many of us need high-quality software to go about our daily jobs. Table 1.3 shows typical software usage patterns for a sample of positions that include knowledge work, based on observations and discussions with members of various professions and from studies with the companies that provide the software.

Table 1.3. Occupation Group Software Usage Circa 2011

Occupation Groups

Function Points

Lines of Code

Daily Usage Hours

Packages Used

Military planners

5,000,000

295,000,000

6.50

30

Physicians

3,000,000

177,000,000

3.00

20

FBI agents

1,500,000

88,500,000

3.50

15

Military officers

775,000

45,725,000

3.50

20

Attorneys

350,000

20,650,000

4.00

10

Airline pilots

350,000

20,650,000

7.00

15

Air-traffic controllers

325,000

19,175,000

8.50

3

IRS tax agents

175,000

10,325,000

5.00

10

Accountants

175,000

10,325,000

5.00

12

Pharmacists

150,000

8,850,000

4.00

6

Electrical engineers

100,000

5,900,000

5.50

20

Software engineers

75,000

4,425,000

7.00

20

Civil engineers

65,000

3,835,000

5.00

6

Police detectives

60,000

3,540,000

3.50

12

Project managers

50,000

2,950,000

2.00

7

Real estate agents

30,000

1,770,000

4.00

7

Bank tellers

25,000

1,475,000

6.00

8

School teachers

15,000

885,000

1.50

4

Retail clerks

15,000

885,000

7.00

5

AVERAGES

643,947

37,992,895

4.82

12

As can be seen from Table 1.3, all knowledge workers in the modern world are heavily dependent on computers and software to perform their jobs. Therefore, these same workers are heavily dependent on high software quality levels. Every time there is a computer failure or a software failure, many knowledge workers will have to stop their jobs until repairs are made. Indeed, power failures can stop work in today's world.

One of the authors was once an expert witness in a software breach-of-contract lawsuit. While being deposed in Boston there was a power failure, and the court stenographer could not record the transcript. As a result, four attorneys, the stenographer, and two expert witnesses spent about two hours waiting until the deposition could continue. All of us were being paid our regular rates during the outage. We are so dependent on computers and software that work stops cold when the equipment is unavailable.

Similar occurrences take place after hurricanes and natural disasters that shut down power. Many retail establishments are unable to record sales information, and some stay closed even though workers and potential customers are both available. If computers and software are out of service, many businesses can no longer operate.

Software and computers are so deeply enmeshed in modern business and government operations that the global economy is at serious risk. As military planners know, nuclear explosions in the atmosphere emit an electromagnetic pulse (EMP) that damages transistors and electrical circuits. They can also cause explosions of liquid fuels such as gasoline and can detonate stored weapons.

Such "ebombs" can be designed and detonated high enough so that they don't cause injuries or death to people, but instead cause major destruction of electronic devices such as radar, electric power generation, television, computers, and the like.

As of 2011, it is thought that most major countries already have ebombs in their arsenals. CBS news reported that one or more ebombs shut down the electric capacity of Baghdad without doing physical damage to buildings or personnel during the second Iraq war. This could be one of the reasons why restoring power to Baghdad after the hostilities ended has been so difficult.

A final reason that software quality is important is because dozens of government agencies and thousands of companies have personal information about us stored in their computers. Therefore, both quality and security are critical topics in 2011.

Table 1.4 shows examples of the kinds of organizations that record personal information and the probable number of people who work in those organizations who might have access to data about our finances, our Social Security numbers, our health-care records, our dates of birth, our jobs, our families, our incomes, and many other personal topics.

Table 1.4. Estimated Applications with Personal Data

Organizations

Function Points

Lines of Code

Personnel with Access

Packages Used

Internal Revenue Service

150,000

7,500,000

10,000

10

Banks

125,000

6,250,000

90,000

12

Insurance companies

125,000

6,250,000

75,000

15

Credit card companies

125,000

6,250,000

3,000

10

Credit bureaus

120,000

6,000,000

1,500

9

Census Bureau

100,000

5,000,000

1,000

5

State tax boards

90,000

4,500,000

200

5

Airlines

75,000

3,750,000

250

12

Police organizations

75,000

3,750,000

10,000

5

Hospitals

75,000

3,750,000

1,000

5

Web-based stores

75,000

3,750,000

1,500

12

Municipal tax boards

50,000

2,500,000

20

3

Motor vehicle department

50,000

2,500,000

200

3

Physicians offices

30,000

1,500,000

50

6

Dental offices

30,000

1,500,000

50

6

Schools/universities

25,000

1,250,000

125

8

Clubs and associations

20,000

1,000,000

250

3

Retail stores

20,000

1,000,000

100

4

TOTALS

1,360,000

68,000,000

194,245

133

Given the number of government agencies and corporations that record vital data about citizens, and the number of people who have access to that data, it is no wonder that identity theft is likely to hit about 15% of U.S. citizens within the next five years.

A Congressional report showed that the number of U.S. cyber attacks increased from about 43,000 in 2008 to more than 80,000 in 2009. As this book is being written, probably more than 10,000 U.S. hackers are actively engaged in attempting to steal credit card and financial information. Computers, networks, and smart phones are all at considerable risk. Security vulnerabilities are linked closely to poor quality, and many attacks are based on known quality flaws.

Because computers and software are now the main tools that operate industry and government, software quality and software security are among the most important topics of the modern world. Indeed, the importance of both quality and security will increase over the next decade.

From an economic standpoint, higher software quality levels can shorten development schedules, lower development and maintenance costs, improve customer satisfaction, improve team morale, and improve the status of the software engineering profession all at the same time.

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020