Improving Software Economics, Part 6 of 7: Achieving 'Agility at Scale': Top 10 Principles of Agile Software Delivery
After 10 years of experience with iterative development projects, we have experience from hundreds of projects to update our management principles. The transitional mix of disciplines promoted in iterative development needs to be updated to the more advanced economic disciplines of Agile software delivery. What follows is my proposed top 10 principles for achieving Agile software delivery success.
Top 10 Management Principles of Agile Software Delivery
- Reduce uncertainties by addressing architecturally significant decisions first.
- Establish an adaptive lifecycle process that accelerates variance reduction.
- Reduce the amount of custom development through asset reuse and middleware.
- Instrument the process to measure cost of change, quality trends, and progress trends.
- Communicate honest progressions and digressions with all stakeholders.
- Collaborate regularly with stakeholders to renegotiate priorities, scope, resources, and plans.
- Continually integrate releases and test usage scenarios with evolving breadth and depth.
- Establish a collaboration platform that enhances teamwork among potentially distributed teams.
- Enhance the freedom to change plans, scope, and code releases through automation.
- Establish a governance model that guarantees creative freedoms to practitioners.
Successfully delivering software products in a predictable and profitable manner requires an evolving mixture of discovery, production, assessment, and a steering leadership style. The word steering implies active management involvement and frequent course-correction to produce better results. All stakeholders must collaborate to converge on moving targets, and the principles listed above delineate the economic foundations necessary to achieve good steering mechanisms. Figure 1 illustrates three important conclusions that can be derived from these principles and practical experience.
Figure 1 The governance of Agile software delivery means managing uncertainty and variance through steering.
In a healthy software project, each phase of development produces an increased level of understanding in the evolving plans, specifications, and completed solution, because each phase furthers a sequence of executable capabilities and the team's knowledge of competing objectives. At any point in the lifecycle, the precision of the subordinate artifacts should be in balance with the evolving precision in understanding, at compatible levels of detail and reasonably traceable to each other.
The difference between precision and accuracy in the context of software management is not trivial. Software management is full of gray areas, situation dependencies, and ambiguous tradeoffs. Understanding the difference between precision and accuracy is a fundamental skill of good software managers, who must accurately forecast estimates, risks, and the effects of change. Precision implies repeatability or elimination of uncertainty. Unjustified precisionin requirements or planshas proved to be a substantial yet subtle recurring obstacle to success. Most of the time, this early precision is just plain dishonest and serves to provide a counter-productive façade for portraying illusory progress and quality. Unfortunately, many sponsors and stakeholders demand this early precision and detail because it gives them (false) comfort of the progress achieved.
Iterative development processes have evolved into more successful Agile delivery processes by improving navigation through uncertainty with balanced precision. This steering requires dynamic controls and intermediate checkpoints, whereby stakeholders can assess what they have achieved so far, what perturbations they should make to the target objectives, and how to refactor what they have achieved to adjust and deliver those targets in the most economical way. The key outcome of these modern Agile delivery principles is increased flexibility, which enables the continuous negotiation of scope, plans, and solutions for effective economic governance.
Figure 2 provides another example of this important metric pattern. This figure illustrates the tangible evolution of a quality metricin this case, the demonstrated mean time between failures (MTBF) for the software embedded in a large-scale command-and-control system.  Whereas the conventional process would have to deal speculatively with this critical performance requirement for most of the lifecycle, the project that employs a modern Agile delivery approach eliminates the uncertainty in achieving this requirement early enough in the project's schedule that the team can effectively trade off remaining resources to invest in more runtime performance, added functionality, or improved profit on system delivery. This sort of reduction in uncertainty has significant economic leverage to all stakeholders.
Figure 2 Reduced uncertainty in critical quality requirements improves the variance in the cost to complete and adds flexibility in downstream resource investments.
I have observed four discriminating patterns that are characteristic of successful Agile delivery projects. These patterns represent a few "abstract gauges" that help the steering process to assess scope management, process management, progress management, and quality management. My hunch is that most project managers certified in traditional engineering project management will react negatively to these notions, because they run somewhat counter to conventional wisdom.
- Scope evolves: Solutions evolve from stakeholder needs, and stakeholder needs evolve from available solutions assets. [Anti-pattern: Get all the requirements right up front.] This equal and opposite interaction between user need and solution is the engine for iteration that is driving more and more asset-based development. We just don't build many applications dominated by custom code development anymore. A vision statement evolves into interim evaluation criteria, which evolve into test cases and finally detailed acceptance criteria. Scope evolves from abstract and accurate representations into precise and detailed representations as stakeholder understanding evolves (i.e., uncertainty is reduced).
- Process rigor evolves: Process and instrumentation evolve from flexible to rigorous as the lifecycle activities evolve from early, creative tasks to later production tasks. [Anti-pattern: Define the entire project's lifecycle process as light or heavy.] Process rigor should be much like the force of gravity: The closer you are to a product release, the stronger the influence of process, tools, and instrumentation on the day-to-day activities of the workforce. The farther you are from a release date, the weaker the influence. This is a key requirement to be fulfilled by the development platform with automation support for process enactment if practitioners are to perceive a lifecycle process that delivers "painless governance."
- Progress assessment is honest: Healthy projects display a sequence of progressions and digressions. [Anti-pattern: Consistently progressing to 100% earned value as the original plan is executed, without any noticeable digression until late in the lifecycle.] The transition to a demonstration-driven lifecycle results in a very different project profile. Rather than a linear progression (often dishonest) of earned value, a healthy project will exhibit an honest sequence of progressions and digressions as they resolve uncertainties, refactor architectures and scope, and converge on an economically governed solution.
- Testing is the steering mechanism: Testing of demonstrable releases is a full lifecycle activity, and the cost of change in software releases improves or stabilizes over time. [Anti-pattern: Testing is a subordinate, bureaucratic, late lifecycle activity, and the cost of change increases over time]. Testing demands objective evaluation through execution of software releases under a controlled scenario with an expected outcome. In an Agile delivery process that is risk-driven, integration testing will mostly precede unit testing and result in more flexibility in steering, with more favorable cost of change trends.
With immature metrics and measures, software project managers are still overly focused on playing defense and struggling with subjective risk management. With further advances in software measurement and collaborative platforms that support process enactment of best practices and integrated metrics collection and reporting, we can manage uncertainty more objectively. Software project managers can invest more in playing offense through balancing risks with opportunities, and organizations can better exploit the value of software to deliver better economic results in their businesses.