Using Metrics to Improve the Process
Once we began the triage process, we were able to collect data on how the process was working. We started by tracking where development time was spent. Each developer would log his or her time by ticket, which made it easy for us to see time spent by project, by priority, by ticket type, and by release. This feedback mechanism let us know which projects were getting the most attention and which were being slighted.
We looked at the number of tickets created or resolved in a project, how many commits were done for a project, and the number of developers working tickets in a project. By examining where the activity was taking place, we were able to better understand where we might need to focus refactor efforts, spend more attention on testing, or be more careful when pulling together final release information.
We also examined what we were not doing. We looked at the data to understand where the least amount of churn was. We looked at ticket aging reports. We looked at projects that didn't get many hours logged. In the short term, when focusing on triage, such decisions may make sense. But we didn't want to get trapped into short-term thinking. We also wanted to be sure that ignoring certain areas of the code or certain projects was a conscious decision, not an accident.
Since our development team was using Scrum, at the end of each sprint (well, most sprints), we also tried to provide details about which issues were resolved in that sprint. At the same time we showed off features, we tried to show how much work was done to resolve issues from production, or issues that got pulled into the sprint via the triage process. We provided process metrics to make sure that we were focused on the right problems.