- Regression Tests
- Software Support for Regression Testing
- User Interface Testing
- Conclusion
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
User Interface Testing
Unit testing is only part of the overall picture. It's great for back ends and server-side applications that don't have user interfaces. User interface testing is a lot harder, because correctness depends on what the user sees, not what value is computed. A lot of little but annoying errors that can't be detected by unit testing creep into programs: components that fail to resize properly, drawing that happens too often or not at all, programs that don't respond in a timely fashion (or at all).
User interface testing is also harder than unit testing because the tests are difficult to maintain. Every time you alter the user interface by adding or rearranging components, you have to modify the test case, which is never a simple process.
User interface testing is best done by hand, with an actual human eye verifying that the results on the screen make sense, but it's unbelievably tedious. JUnit or other unit regression tests can be run weekly, nightly, or hourly, with nothing but computer time spent on the process. The testing manager who has to redo every single test because of a change in a single line of code (as required by proper testing procedure) gradually goes insane.
It's also difficult on the maintainer. To run an interface by hand requires a description of what the user should see when things happen and a detailed list of things to do, and that list usually falls to the maintainer. Every time that list changes, the document must be maintained to match. The document usually contains screen captures of what the program should look like, and that requires running the code, taking a snapshot, and pasting it into the document. Every menu item renamed and every button moved requires a new snapshot.
A number of programs attempt to automate the GUI testing process. Here are a few of them:
qftestJUI—a commercial product from QFS
Abbot—a SourceForge project
Pounder—another SourceForge project
There are also a variety of tools specific to the particular platform, such as XRunner for X Windows–based systems or WinRunner for Windows.
All of these work in basically the same way: You run your program with the GUI tester "listening" for the actions you perform. Different products listen at different levels. Some listen to the AWT event queue; others listen for native Windows or X Windows calls. They record what the screen looks like after each operation. These actions are saved as a script. Later, the script can be played back, and the program will examine whether the screen looks the same.
Unlike unit testing, which has moved into the software development cycle, these tools are actually far more useful in the maintenance phase. Recording the scripts can be tedious, and scripts are often difficult or impossible to edit by hand. Thus, minor changes usually result in rerecording the entire script. Modular script design can minimize the effort required, but it's still an effort.
Personally, I haven't yet found a GUI tester that's robust enough to warrant the work required. I've always depended on my memory and occasional full-on bouts of testing. Unfortunately, such bouts of testing are usually the result of something going wrong, often during a major demo.
The tools are still under development, especially the ones being developed as open source. Keep an eye on them, and give them a try. They may help you out a lot, today or tomorrow.