A Clear Departure from the Past
Not sure where to start, I did something uncharacteristic for me: I directed my efforts and the efforts of the team to focus on documentation. I decided to start with what I felt would be the easiest task: documenting the initial version of what would be our basic testing methodology going forward. Our methodology was integrated into the two-week sprints that the development team was using, and it mapped to a generic release schedule. The core of the methodology was session-based exploratory testing.
For each feature the development team pulled into a sprint, we would create charters for those features during the sprint where the development work was done. Then, in the following sprint, we would execute the charters—capturing our testing notes in the charters as we executed our tests. All the charters and session notes would be kept on the wiki, all in the same space (we called it the "Charter Base"), organized by sprint and by story. If a defect was logged, it would have to include a link to the testing charter that found it. If a defect was retested, it would have to link to the session notes for that testing.
Once we understood how we would handle new-feature execution, I moved up the testing ladder to regression testing. We created a structure within a new QA space on the wiki to organize regression, performance, and smoke tests by core platform component and/or client. We migrated existing content from other parts of the wiki only when we used that content. For different aspects of smoke and regression testing, we asked for feedback on test cases, trying to ensure that we had the right scope for the testing. When we executed regression and smoke tests, we would use simple wiki or Google Doc checklists.
After a couple of months of focusing on the mechanics of session-based testing, we started to shift our focus to higher-order testing concerns, such as project scope, entry and exit criteria, and ensuring that we had the proper resources. The team created its first test plan—a simple document that outlined testing scope, key milestones, and risks. The plan was useful in helping everyone to understand what testing we were doing (and not doing), how long testing would take, and what the constraints to the testing were. At the same time, we began creating taxonomies to support testing for common elements across components and clients.