Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Validation and Testing
Many people don't understand why software is so prone to errors, and simply accept software bugs as a fact of life. But the same people who accept their PC rebooting twice a day would not tolerate picking up the phone and not receiving a dial tone because of a software bug in the telephone company's switch. You can easily think of other examples of mission-critical software, such as control software for nuclear power plant operations, or the software in your car's antilock braking computer. Even state-of-the-art development techniques can't eliminate 100% of software bugs, but proper software validation and testing can go a long way toward detecting most bugs before software is released to end users. This section discusses some of the common types of testing performed during the software development lifecycle.
Unit Testing
Unit testing is the testing of a single software module, usually developed by a single developer. In most organizations, unit testing is the responsibility of the software developer.
Subsystem and System Testing
An application typically is made up of one or more software modules. System testing tests all software modules in an application. On larger applications, subsystem testing may precede this stage. One of the focuses of subsystem and system-level testing is to test all the interactions between modules.
Black-Box and White-Box Testing
Two different approaches to developing software tests are the black-box and white-box test design methods. Black-box test design treats the software system as a "black box," and doesn't explicitly use knowledge of the internal software structure. Black-box test design is usually described as focusing on testing functional requirements. Synonyms for black-box include behavioral, functional, opaque-box, and closed-box. White-box test design, on the other hand, allows peeking "inside the box," or software component, and focuses specifically on using internal knowledge of the software to guide the selection of test data. Synonyms for white-box include structural, glass-box, and clear-box.
While black-box and white-box are terms that are still in popular use, many people prefer the terms behavioral and structural, respectively, for these two testing designs. Behavioral test design is slightly different from black-box test design because the use of internal knowledge isn't strictly forbidden, but simply discouraged.
In practice, it hasn't proven helpful to use a single test design method. Many organizations use a mixture of different methods so that they aren't hindered by the limitations of a particular approach.
It's important to understand that these methods are used during the test design phase. The influence of the test design method is hard to see once the tests are implemented. Note that any level of testing (unit testing, system testing, and so on) can use any test design method(s). Unit testing is usually associated with structural test design, simply because the developer designing a unit test is typically the same person who wrote the code. Subsystem and system-level tests are more likely to use behavioral test design methods.
Alpha and Beta Testing
Alpha testing refers to internal company testing of an application prior to its external release. External users prior to official release of the software typically do beta testing. In both cases, users exercise the application software for its intended purpose and report any bugs they may encounter. Many companies have found both alpha and beta testing extremely useful because they allow much more extensive testing than could be accomplished solely by the in-house development and quality assurance teams.
Several years ago, Sun Microsystems began an extensive alpha test program of its Solaris operating system. Literally hundreds of engineers throughout the company who were unrelated to the actual operating system development installed early builds of Solaris, often six months or more before customer release. This testing was so successful that the Solaris group took the next step and began installing weekly alpha release updates on their main file server machine, providing production service to more than 400 engineers. It certainly doesn't take long to get bugs fixed in that environment. The Solaris alpha test program was so successful that many other software product groups within Sun are now alpha testing their software on internal engineering desktops and servers.
Beta testing may also have the advantage of providing users with early access to new features, and thus building or maintaining a customer base in rapidly changing markets such as the Internet. Netscape has certainly been one of the most widespread sponsors of beta test programs. Some of Netscape's web browser products have undergone a half-dozen or more beta releases, with millions of users. Before widespread use of the Internet, such feedback was impossible simply because of distribution issues. Netscape can beta test six releases, one per week, of its software—with a million or more Internet downloads for each release. Just to produce and distribute a million CD-ROMs would take six weeks for most software vendors.
Stress Testing
The purpose of stress testing is to ascertain that application software will meet its design requirements even under full performance loads and under all possible input conditions. Because software performance is so closely tied to system hardware, stress testing most often is accomplished on the actual production hardware or a replica thereof.