- Reasons for Not Fixing a Bug
- Effective Bug Reporting
- About This Article
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Effective Bug Reporting
As in the case with Chicken Little, running around screaming that the sky is falling usually isn't an effective approach for communicating a problem (unless, of course, the sky really is falling and it's obvious that it is). Most bugs that you find won't be as dramatic as this. They will require you to clearly and succinctly communicate your findings to the team making the fix/no-fix judgment so that they have all the information they need to decide what to do.
You don't need to be a lawyer or an ex-debate team captain to know how to persuade everyone that your bugs need to be fixed. Common sense and basic communication skills will take you a long way. Consider these fundamental principles for reporting a bug:
-
Report bugs as soon as possible. This has been discussed many times before, but it can't be emphasized enough. The earlier you find a bug, the more time that remains in the schedule to get it fixed. Suppose that you find an embarrassing misspelling in a Help file a few months before the software is released. That bug has a very high likelihood of being fixed. If you find the same bug a few hours before the release, odds are it won't be fixed. Figure 1 shows this relationship between time and bug fixing on a graph.
-
This may seem strange—the bug is still the same bug whether you find it today or three months from now. Ideally, it shouldn't matter when it's found, just what the bug is. In reality, however, the risks of fixing that bug go up over time and increasingly weigh on the decision-making process.
-
Effectively describe the bugs. Suppose that you were a programmer and received the following bug report from a tester: "Whenever I type a bunch of random characters in the login box, the software starts to do weird stuff." How would you even begin to fix this bug without knowing what the random characters were, how big a bunch is, and what kind of weird stuff was happening?
-
Minimal. It explains just the facts and the details necessary to demonstrate and describe the bug. Stating "a bunch of random characters" isn't minimal. Give an exact sequence of steps that shows the problem. If more than one set of inputs or actions causes the bug, cite a couple of examples, especially if they show a pattern or a clue that might help the programmer find the cause. Be short and to the point.
-
Singular. There should be only one bug per report. This sounds obvious, but it's sometimes difficult to differentiate between similar-looking bugs and in the rush to get a product out it may seem easier to just lump them together. The problem with reporting more than one bug in a single report is that it's usually only the first bug that gets attention and is fixed—the others become forgotten or ignored. It's also impossible to individually track multiple bugs that are listed on the same report.
It's easy to say that bugs should be reported individually and not grouped together, but it's not always straightforward to do. Consider this bug report: "The following 15 words are misspelled in the online help file: …." That, obviously, should be reported as 15 separate bugs. But what about, "The login dialog won't accept passwords or login IDs with uppercase characters"? Is that one bug or two? From the user perspective, it looks like two, one against the passwords and another against the login IDs. But, at the code level, it may be just one where the programmer didn't handle uppercase characters correctly.
A quick tip: When in doubt, enter individual bugs. You're looking for symptoms, not causes. Several bugs may turn out to have the same cause, but you can't know that until the bug is fixed. It's better to err on the side of individual reports than delay or—worse—forget about fixing a bug because it was lumped in with others. -
Obvious and general. A bug described with numerous complex, convoluted steps that shows a very specific instance of a bug is less likely to get fixed than one described with easily performed steps that shows the bug to be very general and readily seen by a user.
Reporting bugs that your test tools or automation finds is a good example of this. Your automation may have run for six hours before finding a bug. A project manager deciding on a bug would be hesitant to fix one that takes six hours of constant keyboard pounding to occur. If you spend some time analyzing the results of your tool, however, you could find that it doesn't take six hours—it just takes 10 common and likely keystrokes. This process is known as isolating a bug. Your automation just happened to stumble upon those keystrokes while it ran. If you want this bug to get serious attention, your bug report should list those 10 magic keystrokes, not the thousands that the automation ran to get there. -
Reproducible. To be taken seriously, a bug report must show the bug to be reproducible—following a predefined set of steps will cause the software to achieve the same state and the bug to occur again. One of the more difficult, but fun, areas of software testing is trying to isolate and reproduce what seems like random software behavior—occasional crashes, chance data corruption, and so on. As soon as you reproduce the bug to its obvious and general steps, you can report it.
-
Be nonjudgmental in reporting bugs. It's easy for testers and programmers to form an adversarial relationship. Bug reports can be viewed by the programmers and others on the development team as the tester's "report card" on their work so they need to be nonjudgmental, nonpersonal, and noninflammatory. A bug report that says, "Your code for controlling the printer is terrible, it just plain doesn't work. I can't believe that you even checked it in for testing," would be out of line. Bug reports should be written against the product, not the person, and state only the facts. No gloating, no grandstanding, no personalizing, no egos, no accusations. Tact and diplomacy are what matters.
-
Follow up on your bug reports. One thing worse than not finding an important bug is finding a bug, reporting it, and then forgetting about it or losing track of it. You've learned that testing software is hard work, so don't let the results of your labor, the bugs you find, become neglected. From the moment you find a bug, it's your responsibility to make sure that it's reported properly and given the attention that it needs to be addressed. A good tester finds and logs lots of bugs. A great tester finds and logs lots of bugs but also continues to monitor them through the process of getting them fixed.
){kind=link}
The later a bug is found, the less likely it is to be fixed, especially if it's a very minor bug.
An Effective Bug Description
Effective bug descriptions would be as follows:
These principles—report bugs as soon as possible, effectively describe them, be nonjudgmental in reporting them, and follow up on them—should be common sense. You could apply these rules to almost any communications task. It's sometimes difficult, though, in the rush to create a product to remember to apply them to your testing. However, if you want to be effective at reporting your bugs and getting them fixed, these are fundamental rules to follow.