The debate rages on about whether traditional approaches to software development are the best for a project, or whether the more recent agile approaches are better. I’ve always thought that the debate is the wrong one to have in the first place: pick your favorite lifecycle, and chances are good that the selection is moot anyways. Most projects, regardless of the lifecycle selection, are run in a way that the uncertainty and risk will overwhelm whether you have decided waterfall or scrum. At the risk of alienating zealots from all camps, here’s why.
Back in the day, traditional approaches advocated a waterfall approach to software development (never mind whether or not you still think that way – in some ways, I do – but bear with me on this). A big monolithic analysis phase, followed in succession by design, implementation and validation. Great in textbooks, but in practice it never goes that smoothly. Change happens, it needs to be accommodated. History has shown that many of these projects don’t go as well as expected: you could take it from the Standish Group that uses their stats to sell their services, or you could base it on personal experience. I know I can.
Dig a bit deeper, though, and we can start to understand that a big requirements stage doesn’t necessarily mean an effective requirements stage. Indeed, it is rarely the case that projects move ahead with an effective common understanding of required scope, it is more often a period of wandering about and asking the customer what they want, then timing out, signing a big fat document and finally getting on with the ‘real work’. Even a first draft of the SWEBOK asserted that requirements analysis is an overhead activity on projects (they got a real earful for that one!). I certainly know now that I had the title of analyst long before I had the proper skills to extract and analyze the right information from stakeholders. Excelerator and Software Through Pictures and Rational Rose helped me make syntactically correct drawings that were really irrelevant in the context of the problem space.
It is a rare university that actually has a full course on requirements in their CS or Software Engineering curriculum. That’s not what the industry is looking for in grads, so the inertia continues. We learn primarily from experience and our peers, working in companies that have traditionally muddled through this phase, and we learn and reinforce substandard approaches. That’s just the way it is, we say. With this less-than-stellar definition of scope that we’ve taken some time to build, struggling along the way, we then move on to a similarly managed design stage, then finally get rolling as we come into the niche we were actually trained in and implement the system. Most of the grief we encounter here actually has its basis in the earlier design and analysis stages, but we forge ahead, and finally ship something through heroic efforts, often late, with less scope, and to a quality level of, well, since we didn’t quantify our quality objectives, we get what we asked for.
In some ways, the whole agile movement is a big backlash against this ‘traditional approach’. Rather than spend all that time up front producing a sub-standard specification that will be subject to massive change anyways, let’s get busy building things in a way that we’re good at, and learn and refine based on our mistakes. Makes a lot of sense, in some ways. Clip off that big fuzzy front-end, use early validation of builds with our customer to converge on the end product. From that perspective alone, there is a good chance that selecting this lifecycle can give you shorter schedules than the traditional approaches.
Where this theory falls down, though, is the expectation that software development is a necessarily un-deterministic process – that discovery is rampant in all projects throughout the lifecycle, so agile approaches will be better in all cases. I strongly disagree. I think most projects are rife with discovery throughout because even if we spend a lot of time up front, we’re not very effective at it. While the practices of strong analysis have been understood for years, they are rarely applied effectively. Even more rare is the conscious consideration of which of these practices will provide the most value on a given project. Most projects I have seen, big or small, innovative or not, would have benefited from a more focused analysis of needs from all stakeholder communities.
Indeed, many of the successes of agile approaches stem from the fact that they are simply removing a big chunk that added little value, and saved some time overall. Agility has tempered expectations, but has not effectively managed them. They have cut waste, but have actually increased uncertainty and risk on their projects without recognizing it. Their implementation stage now takes the lion’s share of the overall lifecycle, but that stage is rife with as much discovery as in the traditional approaches. Arguably more discovery, as any value that the analysis stage would have brought has been removed, injecting more discovery into that implementation stage. But that’s our comfort zone, so we live with it, you may say. Problem is, that there have been projects where that additional discovery has overwhelmed any cost savings gained through elimination of an explicit analysis stage. Embracing change has become a nice way of saying we are going to live with even more uncertainty and risk than before, because we have thrown out the analysis baby with the bathwater.
And lo, development teams and agile consultants rejoiced everywhere. But sorry, businesses still aren’t getting the level of value they deserve.
Just because we’re pretty bad at doing something doesn’t mean we should stop doing it. While that is the easy way out (and a great selling feature), it is far more effective to improve our approaches for analysis, and to recognize that every project merits a deep common understanding to some degree. That point is deeper on most projects than we typically run with, and the value of doing so is the dramatic reduction of risk and uncertainty downstream on our projects. If you have selected your lifecycle before you made it to this stage, that selection has been made with insufficient information. Are you confident enough in your approach to developing software, traditional or agile (if you feel you must be in a camp), to bid fixed-price on a project after a brief analysis of the client’s needs? Do you think you can make money with this approach?
Most teams will shy away from this, suggesting that software is inherently tough to characterize in that way. While some innovative projects bring with them a large amount of risk, I think it is time to stop making excuses, recognize that almost all projects are run with an unacceptable amount of risk and uncertainty, and work to consciously reduce that risk and uncertainty early in the lifecycle. No matter which lifecycle floats your boat, you owe it to your stakeholders to get more effective at understanding their needs as early as possible. Trust me, you’ll stand out above the silly lifecycle arguments if you do so.
Take advantage of special member promotions, everyday discounts, quick access to saved content, and more! Join Today.