- An Elusive Bug
- Analysis
- Exploratory Testing on Agile Projects Can Be a Good Fit
- Exploratory Testing Is Highly Adaptive
- When To Apply Exploratory Testing
- For More Information
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Exploratory Testing Is Highly Adaptive
Exploratory testing allows for learning by adapting test designs as new information is discovered. Jason and I adapted our tests based on the application information we were getting. This method helped us narrow down a cause of a sporadic bug quickly. If we had pre-scripted our tests, we wouldn't have found it within the hour of time we had. We also wouldn't have predicted the human element of our own assumptions and those of the various teams in our test design. This was something we observed, and we adapted our testing to confirm what we thought was occurring.
James Bach has said that scripted testing emphasizes things like predictability and decidability, while exploratory testing relies on adaptability, credibility, and learning. We solve testing problems over the course of a project by learning, gathering more information, and utilizing the skills of diverse team members. We don't try to solve all the testing problems up front. We try to minimize documentation (this doesn't necessarily mean no documentation), and maximize the number of tests we can create and execute. As Martin Fowler points out, agile projects tend to be adaptive rather than predictive. Due to the interactivity of exploratory testing and a reliance on rich feedback, skilled exploratory testing works well with an adaptive development model. Conversely, designing all tests "test-first" can influence all test design efforts to be predictive, potentially cutting off interactive, adaptive test design and execution.
Learning is an important aspect of exploratory testing. Agile teams also value learning. As Kent Beck writes in Extreme Programming Explained, "What is it that we want to get out of code? The most important thing is learning. The way I learn is to have a thought, then test it out to see if it is a good thought. Code is the best way I know how to do this." A good exploratory tester feels similarly about testing. The best way to test out a thought is to use the software. We design tests, run them, and observe behavior in an application, often simultaneously. We inquire, make discoveries, and rerun the tests to confirm or contradict our ideas. We constantly challenge our testing ideas and learn about the software we're testing, and about our own mental models we're using to develop tests. Tests developed and executed in this way help provide all kinds of information about a product—not just bug reports.
Exploratory testing is also a practice of rapid software testing, which is taught by James Bach and Michael Bolton. Rapid software testing focuses on testing effectively and providing feedback when the tester has barely sufficient information and is under extreme time pressure. This strategy is useful for agile projects where things can change quickly, especially when iterations may be two weeks long. The tests we develop and automate can only tell us about what the code is doing according to the tests we thought of at the time. Dealing with change can be difficult; our tests might be confirming the wrong behavior. Ward Cunningham provided an answer on the Agile Testing mailing list to the question Why should agile teams do exploratory testing?: "Because an agile development project can accept new and unanticipated functionality so fast, it is impossible to reason out the consequences of every decision ahead of time. In other words, agile programs are more subject to unintended consequences of choices simply because choices happen so much faster. This is where exploratory testing saves the day. Because the program always runs, it is always ready to be explored."