Finding the Right Balance
With all this new documentation to do, we slowed down a little bit, and in most cases that caution helped us to make better decisions. When we first started chartering, we'd get almost a one-to-one ratio of testing charters to sprint features. It was difficult for the team to see a feature and think of a large number of tests. By contrast, a few months into the new process, the team had evolved to a one-to-many relationship, with some features having more than six hours of required testing identified. As the team got more practice, we came up with more and better testing ideas.
Because the rest of the company worked on the wiki, so did we. The wiki is a lighter method of documentation than 15-page word templates, which tends to make wiki documentation less formal and easier to read. However, unlike other teams, we developed some standards for how documentation should look, how it should be structured, and how often the documentation should be reviewed or restructured. This approach decreased our maintenance overhead, and we could still find stuff six months after we made it. It also gave us something simple and preformatted that we could export to PDF for an auditor.
For most technical topics, we also preferred pictures to words. If there was an interaction between systems, the goal was to show—not tell. Most members of the team used Visio or OmniGraffle. In some cases, whiteboard photos were sufficient. The goal was to find the right balance between speed and communication. In many cases, this documentation was the first of its kind for the company.
After a few months of working through some of the various documents, it became clear which types of documentation provided the most value. Going forward, the team decided to continue investing in the following documents:
- Exploratory testing charters and session notes
- Regression test suites (including building out the data test beds to support the test cases)
- Test plans for new clients or large platform changes
- Testing taxonomies for lessons learned or for similar features across components/clients
- Feature wiki pages detailing purpose, technical details, configuration details, and other information as new stories were completed in a sprint
While the team could also see value in other types of documentation, they were always aware of the opportunity cost of documentation. If you're writing documentation, you aren't testing, pairing with someone newer to the team, talking with one of the programmers or someone from operations, or researching new methods of testing. It's important to find the right balance of technical documentation, record of test execution, and planning documentation; but it's also important to make sure that you remain efficient and effective.