Improving the Testing Feedback
At the retrospective meeting, the developers had one complaint: The feedback loop on testing information was too long, because they had to wait until the final Sprint to get feedback on features that were developed sometimes months earlier. When we planned for the next product release, we decided to try a new approach.
The first improvement we made was to test in a cycle one Sprint behind development throughout the project. Once the development group completed and demonstrated the software at the end of a Sprint, we tested those features while the developers got started on the next Sprint. At first this strategy worked okay, but the further we got in the project, the more difficult it became to provide feedback on the emerging design and to test already developed features from prior Sprints at the same time.
As testers, we were much more engaged from the beginning of this second project, since we now had more responsibility during the development instead of waiting until the end. We were more effective at getting information early on because we were accountable to the team on a daily basis and at the end of each Sprint. And the developers were happy to get feedback more quickly—especially bug reports.
Once we reached the final testing and debugging Sprint, we had already tested much of the product's functionality. This took the pressure off because we were familiar with the software we were testing, knowing areas of risk and what was most important to the client. We were able to do much more testing on a project than we had done before, and could focus on risk areas more comfortably. This leeway inspired new test ideas at the end of the project, but unfortunately, we had to trim down what we wanted to test because more testing time simply wasn't available.
While we were confident in the amount of testing we had done, we felt we could do more. This was hard to take. In a typical waterfall process, you simply don't have as much time using the software, so sometimes it's easier to let something go if it isn't tested well. In an iterative process, it's easier to spend more time testing throughout development and spend more time thinking about testing. It's common to generate a lot of testing ideas and feel responsible for areas you may not have the time to reach. We felt a bit of "tester guilt" at not getting to every area we had hoped to test, but still shipped the software with an unprecedented level of confidence.
The developers were pleased with the improvements to the feedback loop, but they still wanted even tighter feedback on defect reporting. On the next project, we added two days of testing at the end of each Sprint prior to the demo. This helped the team have more confidence in the demo, before the customer saw the product, and provided feedback on developed software within the Sprint.