Agile Methods and Traditional Processes
Agile methods differ from traditional industrial processes in a couple of ways. Agile planning differs from traditional planning because agile planning is—to use the words of Captain Barbossa23—“more what you’d call a guideline.” Agile development tends to follow a depth-first approach rather than the breadth-first approach of traditional methods. Another key agile practice is test-driven development (TDD), which pushes testing as far up front in the process as possible. Finally, agile embraces change rather than fearing it.
It is a common and well-known problem in numerical analysis that the precision of a computational result cannot be better than that of the elements used within the computation.24 I have seen schedules for complex system development projects that stretch on for years yet identify the completion time to the minute. Clearly, the level of knowledge doesn’t support such a precise conclusion. In addition (pun intended ), errors accumulate during computations; that is, a long computation compounds the errors of its individual terms.
If you are used to working in a traditional plan-based approach, agile methods may seem chaotic and intimidating. The problem with the standard waterfall style is that although plans may be highly detailed and ostensibly more complete, that detail is wrong and the computed costs and end dates are in error.
Further, not only is the information you have about estimates fuzzy at best, it is also usually systematically biased toward the low end. This is often a result of management pressure for a lower number, with the misguided intention of providing a “sense of urgency” to the developers. Sometimes this comes from engineers with an overdeveloped sense of optimism. Maybe it comes from the marketing staff who require a systematic reduction of the schedule by 20%, regardless of the facts of the matter. In any event, a systematic but uncorrected bias in the estimates doesn’t do anything but further degrade the accuracy of the plan.
Beyond the lack of precision in the estimates and the systematic bias, there is also the problem of stuff you don’t know and don’t know that you don’t know. Things go wrong on projects—all projects. Not all things. Not even most things. But you can bet money that something unexpected will go wrong. Perhaps a team member will leave to work for a competitor. Perhaps a supplier will stop producing a crucial part and you’ll have to search for a replacement. Maybe as-yet-unknown errors in your compiler itself will cause you to waste precious weeks trying to find the problem. Perhaps the office assistant is really a KGB25 agent carefully placed to bring down the Western economy by single-handedly intercepting and losing your office memo.
It is important to understand, deep within your hindbrain, that planning the unknown entails inherent inaccuracy. This doesn’t mean that you shouldn’t plan software development or that the plans you come up with shouldn’t be as accurate as is needed. But it does mean that you need to be aware that they contain errors.
Because software plans contain errors that cannot be entirely removed, schedules need to be tracked and maintained frequently to take into account the “facts on the ground.” This is what we mean by the term dynamic planning—it is planning to track and replan when and as necessary.
If you look at a traditional waterfall approach, such as is shown in Figure 1.10, the process can be viewed as a sequential movement through a set of layers. In the traditional view, each layer (or “phase”) is worked to completion before moving on. This is a “breadth-first” approach. It has the advantage that the phase and the artifacts that it creates are complete before moving on. It has the significant disadvantage that the basic assumption of the waterfall approach—that the work within a single phase can be completed without significant error—has been shown to be incorrect. Most projects are late and/or over budget, and at least part of the fault can be laid at the feet of the waterfall lifecycle.
Figure 1.10 Waterfall lifecycle
An incremental approach is more “depth-first,” as shown in Figure 1.11. This is a “depth-first” approach (also known as spiral development) because only a small part of the overall requirements are dealt with at a time; these are detailed, analyzed, designed, and validated before the next set of requirements is examined in detail.26 The result of this approach is that any defects in the requirements, through their initial examination or their subsequent implementation, are uncovered at a much earlier stage. Requirements can be selected on the basis of risk (high-risk first), thus leading to an earlier reduction in project risk. In essence, a large, complex project is sequenced into a series of small, simple projects. The resulting incremental prototypes (also known as builds) are validated and provide a robust starting point for the next set of requirements.
Figure 1.11 Incremental spiral lifecycle
Put another way, we can “unroll” the spiral approach and show its progress over linear time. The resulting figure is a sawtooth curve (see Figure 1.12) that shows the flow of the phases within each spiral and the delivery at the end of each iteration. This release contains “real code” that will be shipped to the customer. The prototype becomes increasingly complete over time as more requirements and functionality are added to it during each microcycle. This means not only that some, presumably the high-risk or most critical requirements, are tested first, but also that they are tested more often than low-risk or less crucial requirements.
Figure 1.12 Unrolling the spiral
In agile approaches, testing is the “stuff of life.” Testing is not something done at the end of the project to mark a check box, but an integral part of daily work. In the best case, requirements are delivered as a set of executable test cases, so it is clear whether or not the requirements are met. As development proceeds, it is common for the developer to write the test cases before writing the software. Certainly, before a function or class is complete, the test cases exist and have been executed. As much as possible, we want to automate this testing and use tools that can assist in creating coverage tests. Chapter 8 deals with the concepts and techniques for agile testing.
Unplanned change in a project can occur either because of the imprecision of knowledge early in the project or because something, well, changed. Market conditions change. Technology changes. Competitors’ products change. Development tools change. We live in a churning sea of chaotic change, yet we cope. Remember when real estate was a fantastic investment that could double your money in a few months? If you counted on that being true forever and built long-range inflexible plans based on that assumption, then you’re probably reading this while pushing your shopping cart down Market Street in San Francisco looking for sandwiches left on the curb. We cope in our daily lives because we know that things will change and we adapt. This doesn’t mean that we don’t have goals and plans but that we adjust those goals and plans to take change into account.
Embracing change isn’t just a slogan or a mantra. Specific practices enable that embracement, such as making plans that specify a range of successful states, means by which changing conditions can be identified, analyzed, and adapted to, and methods for adapting what we do and how we do it to become as nimble and, well, agile, as possible.
In the final analysis, if you can adapt to change better than your competitors, then evolution favors you.27