Home > Articles > Process Improvement

Defining Software Quality and Economic Value

  • Print
  • + Share This
This introduction to The Economics of Software Quality covers how to define software quality and economic value.
This chapter is from the book

This chapter is from the book

Introduction

This book deals with two topics that have been ambiguous and difficult to pin down for many years: software quality and economic value.

The reason for the ambiguity, as noted in the Preface, is that there are many different points of view, and each point of view has a different interpretation of the terms. For example, software quality does not mean the same thing to a customer as it does to a developer. Economic value has a different meaning to vendors than it has to consumers. For vendors, revenue is the key element of value, and for consumers, operational factors represent primary value. Both of these are discussed later in the book.

By examining a wide spectrum of views and extracting the essential points from each view, the authors hope that workable definitions can be established that are comparatively unambiguous.

Software quality, as covered in this book, goes well beyond functional quality (the sort of thing to which customers might react to in addition to usability and reliable performance). Quality certainly covers these aspects but extends further to nonfunctional quality (how well the software does what it is meant to do) and to structural quality (how well it can continue to serve business needs as they evolve and change as business conditions do).

Why Is Software Quality Important?

Computer usage in industrial countries starts at or before age 6, and by age 16 almost 60% of young people in the United States have at least a working knowledge of computers and software. Several skilled hackers have been apprehended who were only 16 years of age.

The approximate population of the United States in 2010 was about 309,800,135 based on Census Bureau estimates. Out of the total population about 30% use computers daily either for business purposes or for recreational purposes or both; that is, about 92,940,040 Americans are daily computer users.

About 65% of the U.S. population use embedded software in the form of smart phones, digital cameras, digital watches, automobile brakes and engine controls, home appliances, and entertainment devices. Many people are not aware that embedded software controls such devices, but it does. In other words, about 201,370,087 U.S. citizens own and use devices that contain embedded software.

Almost 100% of the U.S. population has personal data stored in various online databases maintained by the Census Bureau, the Internal Revenue Service, state governments, municipal governments, banks, insurance companies, credit card companies, and credit scoring companies.

Moving on to business, data from various sources such as Forbes, Manta, Business Week, the Department of Commerce Bureau of Labor Statistics, and others reports that the United States has about 22,553,779 companies (as of the end of 2010). Of these companies about 65% use computers and software for business operations, retail sales, accounting, and other purposes—so about 14,659,956 U.S. companies use computers and software. (Corporate software usage ranges from a basic spreadsheet up to entire enterprise resource planning [ERP] packages plus hundreds of other applications.)

Based on data from the Manta website, the software deployed in the United States is provided by about 77,186 software companies and another 10,000 U.S. companies that create devices with embedded software. A great deal of embedded software and the device companies themselves have moved to China, Taiwan, Japan, India, and other offshore countries. An exception to offshore migration is the manufacture of embedded software for military equipment and weapons systems, which tends to stay in the United States for security reasons.

The U.S. military services and the Department of Defense (DoD) own and deploy more software than any other organizations in history. In fact, the DoD probably owns and deploys more software than the military organizations of all other countries combined. Our entire defense community is now dependent on software for command and control, logistics support, and the actual operation of weapons systems. Our national defense systems are highly computerized, so software quality is a critical component of the U.S. defense strategy.

Without even knowing it, we are awash in a sea of software that operates most of our manufacturing equipment, keeps records on virtually all citizens, and operates the majority of our automobiles, home appliances, and entertainment devices. Our transportation systems, medical systems, and government operations all depend on computers and software and hence also depend on high software quality levels.

While software is among the most widely used products in human history, it also has one of the highest failure rates of any product in human history due primarily to poor quality.

Based on observations among the authors' clients plus observations during expert witness assignments, the cancellation rate for applications in the 10,000 function point size range is about 31%. The average cost for these cancelled projects is about $35,000,000. By contrast, projects in the 10,000 function point size range that are successfully completed and have high quality levels only cost about $20,000,000.

When projects developed by outsource vendors are cancelled and clients sue for breach of contract, the average cost of litigation is about $5,000,000 for the plaintiff and $7,000,000 for the defendant. If the defendants lose, then awards for damages can top $25,000,000. However because most U.S. courts bar suits for consequential damages, the actual losses by the defendants can be much larger.

Of the authors' clients who are involved with outsourcing, about 5% of agreements tend to end up in court for breach of contract. The claims by the plaintiffs include outright failure, delivery of inoperable software, or delivery of software with such high defect volumes that usage is harmful rather than useful.

As of 2011, the average cost per function point in the United States is about $1,000 to build software applications and another $1,000 to maintain and support them for five years: $2,000 per function point in total. For projects that use effective combinations of defect prevention and defect removal activities and achieve high quality levels, average development costs are only about $700 per function point and maintenance, and support costs drop to about $500 per function point: $1,200 per function point in total.

Expressed another way, the software engineering population of the United States is currently around 2,400,000 when software engineers and related occupations such as systems analysis are considered. On any given day, due to poor quality control, about 1,000,000 of these workers spend the day finding and fixing bugs (and, unwittingly, injecting new bugs as part of the process).

So all of these statistics point to the fact that better software quality control in the forms of defect prevention and more effective defect removal could free up about 720,000 software personnel for more productive work than just bug repairs, easily reducing U.S. software development and maintenance costs by about 50%.

As we show later in the book, the cost savings that result from higher quality are proportional to application size. As software projects grow larger, cost savings from high quality levels increase. Table 1.1 illustrates typical software development costs for low, average, and high-quality software applications.

Table 1.1. Software Costs by Size and Quality Level

(Burdened cost = $10,000 per month)

Function Points

Low Quality

Average Quality

High Quality

10

$6,875

$6,250

$5,938

100

$88,561

$78,721

$74,785

1,000

$1,039,889

$920,256

$846,636

10,000

$23,925,127

$23,804,458

$18,724,012

100,000

$507,767,782

$433,989,557

$381,910,810

The technologies and methods associated with these three quality levels are discussed and illustrated in later sections of this chapter, as are the reasons that large software projects are so risky. Suffice it to say the "high quality" column includes effective defect prevention, effective pretest defect removal such as inspections and static analysis, and much more effective testing than the other columns.

Another major reason that software quality is important is because poor quality can and will affect each citizen personally in unpleasant ways. Every time there is a billing error, every time taxes are miscalculated, every time credit ratings change for incorrect reasons, poor software quality is part of the problem.

Early in 2010, hundreds of computers were shut down and many businesses including hospitals were disrupted when the MacAfee antivirus application mistakenly identified part of Microsoft Windows as a virus and stopped it from loading.

According to the July 25, 2010, issue of Computerworld, the BP drilling platform that exploded and sank had been having frequent and serious computer problems for a month prior to the final disaster. These problems prevented significant quantities of data from being analyzed that might have warned operators in time to shut down the oil pumping operation.

If your automobile braking system does not operate correctly, if a home appliance fails unexpectedly, or if a hospital makes a medical mistake, there is a good chance that poor software quality was part of the problem.

If an airline flight is delayed more than about two hours or if there is a widespread power outage that affects an entire geographic region such as New England, the odds, again, are good that poor software quality was part of the problem.

Because software is such a basic commodity as of 2011, it is useful to start by considering how much software ordinary U.S. citizens own and use. Table 1.2 shows typical software volumes associated with normal living activities.

Table 1.2. Personal Software Circa 2011

Products

Function Points

Lines of Code

Daily Usage Hours

Personal computer

1,000,000

50,000,000

2.00

Automobile

350,000

17,500,000

2.00

Smart appliances

100,000

5,000,000

1.00

Smart phone

25,000

1,250,000

1.50

Social networks

25,000

1,250,000

1.50

Home entertainment

10,000

500,000

2.00

Electronic book

5,000

250,000

1.00

Digital camera

2,500

125,000

0.50

Digital watch

1,500

75,000

0.50

TOTALS

1,519,000

75,950,000

12.00

The data in Table 1.2 comes from a combination of web sources and proprietary data provided by clients who build appliances of various kinds.

Not every citizen has all of these appliances and devices, but about half of us do. Many of us have even more than what Table 1.2 indicates, such as owning several automobiles, several cell phones, and numerous appliances. Software quality is important because it is the main operating component of almost all complex machines as of 2011.

Another reason that software quality is important is because many of us need high-quality software to go about our daily jobs. Table 1.3 shows typical software usage patterns for a sample of positions that include knowledge work, based on observations and discussions with members of various professions and from studies with the companies that provide the software.

Table 1.3. Occupation Group Software Usage Circa 2011

Occupation Groups

Function Points

Lines of Code

Daily Usage Hours

Packages Used

Military planners

5,000,000

295,000,000

6.50

30

Physicians

3,000,000

177,000,000

3.00

20

FBI agents

1,500,000

88,500,000

3.50

15

Military officers

775,000

45,725,000

3.50

20

Attorneys

350,000

20,650,000

4.00

10

Airline pilots

350,000

20,650,000

7.00

15

Air-traffic controllers

325,000

19,175,000

8.50

3

IRS tax agents

175,000

10,325,000

5.00

10

Accountants

175,000

10,325,000

5.00

12

Pharmacists

150,000

8,850,000

4.00

6

Electrical engineers

100,000

5,900,000

5.50

20

Software engineers

75,000

4,425,000

7.00

20

Civil engineers

65,000

3,835,000

5.00

6

Police detectives

60,000

3,540,000

3.50

12

Project managers

50,000

2,950,000

2.00

7

Real estate agents

30,000

1,770,000

4.00

7

Bank tellers

25,000

1,475,000

6.00

8

School teachers

15,000

885,000

1.50

4

Retail clerks

15,000

885,000

7.00

5

AVERAGES

643,947

37,992,895

4.82

12

As can be seen from Table 1.3, all knowledge workers in the modern world are heavily dependent on computers and software to perform their jobs. Therefore, these same workers are heavily dependent on high software quality levels. Every time there is a computer failure or a software failure, many knowledge workers will have to stop their jobs until repairs are made. Indeed, power failures can stop work in today's world.

One of the authors was once an expert witness in a software breach-of-contract lawsuit. While being deposed in Boston there was a power failure, and the court stenographer could not record the transcript. As a result, four attorneys, the stenographer, and two expert witnesses spent about two hours waiting until the deposition could continue. All of us were being paid our regular rates during the outage. We are so dependent on computers and software that work stops cold when the equipment is unavailable.

Similar occurrences take place after hurricanes and natural disasters that shut down power. Many retail establishments are unable to record sales information, and some stay closed even though workers and potential customers are both available. If computers and software are out of service, many businesses can no longer operate.

Software and computers are so deeply enmeshed in modern business and government operations that the global economy is at serious risk. As military planners know, nuclear explosions in the atmosphere emit an electromagnetic pulse (EMP) that damages transistors and electrical circuits. They can also cause explosions of liquid fuels such as gasoline and can detonate stored weapons.

Such "ebombs" can be designed and detonated high enough so that they don't cause injuries or death to people, but instead cause major destruction of electronic devices such as radar, electric power generation, television, computers, and the like.

As of 2011, it is thought that most major countries already have ebombs in their arsenals. CBS news reported that one or more ebombs shut down the electric capacity of Baghdad without doing physical damage to buildings or personnel during the second Iraq war. This could be one of the reasons why restoring power to Baghdad after the hostilities ended has been so difficult.

A final reason that software quality is important is because dozens of government agencies and thousands of companies have personal information about us stored in their computers. Therefore, both quality and security are critical topics in 2011.

Table 1.4 shows examples of the kinds of organizations that record personal information and the probable number of people who work in those organizations who might have access to data about our finances, our Social Security numbers, our health-care records, our dates of birth, our jobs, our families, our incomes, and many other personal topics.

Table 1.4. Estimated Applications with Personal Data

Organizations

Function Points

Lines of Code

Personnel with Access

Packages Used

Internal Revenue Service

150,000

7,500,000

10,000

10

Banks

125,000

6,250,000

90,000

12

Insurance companies

125,000

6,250,000

75,000

15

Credit card companies

125,000

6,250,000

3,000

10

Credit bureaus

120,000

6,000,000

1,500

9

Census Bureau

100,000

5,000,000

1,000

5

State tax boards

90,000

4,500,000

200

5

Airlines

75,000

3,750,000

250

12

Police organizations

75,000

3,750,000

10,000

5

Hospitals

75,000

3,750,000

1,000

5

Web-based stores

75,000

3,750,000

1,500

12

Municipal tax boards

50,000

2,500,000

20

3

Motor vehicle department

50,000

2,500,000

200

3

Physicians offices

30,000

1,500,000

50

6

Dental offices

30,000

1,500,000

50

6

Schools/universities

25,000

1,250,000

125

8

Clubs and associations

20,000

1,000,000

250

3

Retail stores

20,000

1,000,000

100

4

TOTALS

1,360,000

68,000,000

194,245

133

Given the number of government agencies and corporations that record vital data about citizens, and the number of people who have access to that data, it is no wonder that identity theft is likely to hit about 15% of U.S. citizens within the next five years.

A Congressional report showed that the number of U.S. cyber attacks increased from about 43,000 in 2008 to more than 80,000 in 2009. As this book is being written, probably more than 10,000 U.S. hackers are actively engaged in attempting to steal credit card and financial information. Computers, networks, and smart phones are all at considerable risk. Security vulnerabilities are linked closely to poor quality, and many attacks are based on known quality flaws.

Because computers and software are now the main tools that operate industry and government, software quality and software security are among the most important topics of the modern world. Indeed, the importance of both quality and security will increase over the next decade.

From an economic standpoint, higher software quality levels can shorten development schedules, lower development and maintenance costs, improve customer satisfaction, improve team morale, and improve the status of the software engineering profession all at the same time.

  • + Share This
  • 🔖 Save To Your Account