This chapter is from the book
This chapter is from the book
This chapter is from the book
3.6 Process and Role Overlap
What if your plumbing contractor started to lay out the pipes before the carpenter had a chance to rough in the walls? What if the electrician decided that he was really the best person to install the windows?
)
Figure
3-1 Planning done out of sequence is wasted effort.2
These analogies illustrate two types of undesirable process overlap that occur commonly in software projects. The first illustrates phase overlap, and the second illustrates role overlap. While some overlap in sharing ideas is good and helpful, the overlap discussed here is counterproductive. Think of it as phase trespassing.
Phase overlap is when one project phase begins before the previous one is completed. Nothing is done well if it is done before its time, before all the information is in and the groundwork has been properly laid. When this happens, the phase begins to produce work based on an incomplete picture. This is especially damaging when the work cannot be corrected later after all the details are known because that would make it embarrassingly apparent that the work was performed prematurely.
Such phase overlap is typically initiated at the urging of impatient managers who do not appreciate that proper timing and sequence are more important than fudging progress and cutting corners by insisting that phases get started before the process is ready.
A slightly different form of overlap is role overlap. Role overlap is when the people working in one phase take on responsibilities best performed in subsequent phases. Software planners at each phase in the process try to do too much. They overstep their bounds and their expertise. A few examples of role overlap include:
-
When the salesman promises a solution strategy
-
When the business analyst designs detailed prototype screens
Flashback—Everyone Is a Designer
Our committee was discussing the content to include in various planning documents. One of the members felt that user interface screens should be finalized early in the process, long before all the specifications had been accumulated. I suggested tactfully that this activity would be better deferred until later in the process. At the very least, I suggested, they should be treated as sketches to create expectation that they will be optimized and finalized later.
My colleague disagreed strongly. He said that he was the person best qualified to design the user interfaces and that he would not want to trust that to someone later in the process, even to make revisions.
Several weeks later I was talking to another colleague who inherited a project. He was disappointed and half-joking about how poorly designed and poorly optimized the application screens were. He then mentioned that the designer of the screens was the same manager who insisted that he was the best qualified for that task at the committee meeting.
-
When the application requirements include system architecture, object models, and database diagrams
There is a reason that different people have different roles at different times in the project. Sometimes the hardest part of the job is knowing what not to do.
Everyone thinks that he or she is the best qualified to create user interfaces. After all, he or she always finds his or her own designs perfectly intuitive. Therefore, everyone from the user to the salesman to the analyst to the architect likes to think they should specify user interfaces.
Yet very few people really have a talent for this art. When any task, like interface or database design or application architecture, gets done too early, the wrong person using incomplete information and the wrong tool probably does it (Cooper, 1995).
In a true waterfall model, process overlap is a common killer. But it is also a problem in the various iterative models as well. In each iteration, the process should flow sequentially through the logical phases with the right people doing the right task at the right time. The project may be broken up into modules or phases, but each of those subprocesses should respect phase integrity independently.
That is not to imply that each phase works in isolation. Take the example of user interfaces again. It is helpful if the user shows the analyst some current screens. Likewise, it is useful when the analyst sketches some revised designs to accommodate new functionality. It is helpful when those sketches are optimized during the requirements specification as mock-ups. Mock-ups are consolidated and prototyped during design. Those prototypes can be adapted to better meet actual ergonomics during the development phase. When those resulting screens are found to be nonintuitive during testing, they should be re-engineered.
Respecting phase boundaries does not mean staking out exclusive claim to tasks. The key point here is that no one phase should proclaim perfection and freeze a design. As long as subsequent phases have the freedom to refine the designs, the input of previous phases is welcome and beneficial. It is only when specifications are locked that phase overlap becomes a critical problem.
Another form of process overlap occurs when a task is corrected downstream rather than being sent back upstream. If the developer is handed an incomplete specification, he or she should pass that back to the planner rather than become a planner. The developer should be allowed and expected to make refinements, but should not redesign an architecture that is flawed. Instead, it should go back to the architect for rework.
The best way to discourage phase overlap is to set the expectation that all work is subject to change downstream or subject to being sent back upstream. This ensures that the best decisions are made by the appropriate people at the optimal time.
Following is a summary of the various activities that should be encouraged or discouraged during each of the project phases to avoid unwanted overlap problems.
Sales Phase
The important thing to strive for during the initial meetings is to avoid artificially constraining options. Don't set expectations during the earliest contacts that will become a ball and chain on the leg of the entire subsequent planning and development effort. Don't promise specific technologies or approaches except where explicitly defined by the client as key requirements.
The worst thing that can happen is that the cost, scope, and duration of the project are all fixed in the mind of the client during the initial sales meetings. When this happens, you will almost certainly get the contract, but it is also as certain that the project will be a bloody death march.
Here is a classic example:
“Of course we'll have to do more analysis, but we can almost certainly replace and update your legacy system and get it deployed by year-end within your budget.”
This promise constrains all three sides of the project triangle. The budget cap is fixed. The delivery date is fixed. And the scope is not only fixed, but is open-ended. No matter what depth and scope of functionality are subsequently uncovered in the legacy system, the expectation has been established that it will be replicated. Despite all the disclaimers in the world, the client will still base their level of satisfaction on this expectation. No wonder so many projects are perceived as failures.
It is admittedly difficult for the sales professional to avoid this sort of promise and still close any sales. It is best to focus on past successes to close the deal, rather than get suckered into making overly specific, or even worse, overly general promises.
Your job must be to promise nothing except the integrity, quality, and professionalism of your planning and development team. Stress the planning and development strategies that give your firm a competitive advantage. You must explain to the client how your process works, from envisioning to maintenance. It goes without saying that you should actually have a good planning and development process in place. If effective software planning is all talk on your part, that will become obvious to the client eventually.
And of course, you must still close the deal with the client to proceed to a paid planning phase in spite of these difficult challenges. I'm glad I'm not in sales!
Envisioning Phase
The Vision Document is the accepted deliverable of the envisioning phase. It should provide an overview of the business need and present a vision of a new solution. The most common pitfall is specifying too much and planning prematurely, thereby overly constraining or unnecessarily biasing subsequent efforts.
Using the house analogy, the typical Vision Document might say:
“The current Jones home is too small. A new house will be built to meet the needs of their growing family. It will be constructed of brick with a full basement. It will have a walk-up attic for additional storage and will be ready before the first snowfall. The price will not exceed $250K, including a three-car garage. It should be attractive with efficient traffic flow.”
This vision statement is both too specific and too general. It specifies how the house will be constructed before any detailed analysis. It sets a price expectation while at the same time promising to meet all unspecified needs. A better Vision Document could say:
“The current Jones home is too small. With another child on the way, more room is required. With their parents aging, accommodations for them will be needed in the near future. The current home has only a single bathroom that is insufficient to meet future needs. More storage space is required. The Jones's family has two cars.”
This version states the problem but does not specify the solution. It does not specify nonobjective goals or establish time estimates.
Estimation is the trickiest part of this phase. In most cases, the envisioning phase has much more to do with price negotiation than envisioning. The client is probably paying you to perform a high-level analysis of the project. They expect that at the end of this investment of time, you will be able to provide them with a solid estimate of the entire cost of the project.
You therefore have three major options. The least desirable option is to estimate based on that several-week analysis. This almost always results in a low estimate intended to retain the business. It is typically based more on the client's perceived budget then actual requirements.
A more desirable, but less saleable, option is to come up with a budget figure for the detailed design phase only, responsibly resisting getting locked into a development estimate without a complete design specification. The problem with this approach is that you can't normally get business this way. The client will not be very thrilled about underwriting a six-month planning effort without having any idea of the subsequent development cost. Further, you run the risk of underbidding your planning effort and entering development without a blueprint.
The third option is to suggest a phased approach, where feasible. Break the project up into a series of smaller subprojects so that the client can reduce risk and begin to build confidence in your firm. The risk here is that the client may end up with a poorly integrated suite of applications. If your client is inclined to go this way, try to convince them to at least fund all high-level planning up front and then implement the detailed planning and coding in phases.
This dilemma causes some software shops to provide planning services as a completely separate practice, separating planning from the production side in the expectations of the client. These companies can sell analysis and design services separately. If the client then chooses to have them implement that design, they can do that also. To succeed using this strategy, your challenge is to prove to the client that you can, in fact, provide them with a valuable project plan which could be taken anywhere without redoing much of the design work. Hence, we return to the overriding need for a systematic planning process.
The need for a portable, self-contained plan exists even in integrated planning and development efforts. With today's staff turnover, any business that relies upon the memory of employees to retain planning specifics is doomed to lose at least some of that knowledge before the plan ever gets to development. When that happens, developers must go back to their client repeatedly for the same information. Clients typically aren't very happy about this for long.
Analysis Phase
Once the vision is presented, the analysis phase begins to uncover and document specific requirements. Using the house building analogy, this is the phase during which the designer talks to the prospective owners to learn specifically what they need and would like in their new house.
Again, the most common mistake is premature planning. Instead of limiting the analysis documents to specifying the business requirements, the analyst proceeds into designing the solution. Prototype screens are developed and the expectation is set with the client that they are finalized. Logical and physical architectures are defined. Tools and techniques are specified, and database schemas are created.
All of these things are premature. The solution should not be specified until after all the requirements are known. To do so risks an inefficient and fragmented solution.
“The requirements should also not include information about the system design or architecture. Otherwise, you may accidentally have restricted your team from pursuing whatever design options make the most sense for your application.” (Leffingwell, 2000)3
The other problem is that a good business analyst might not be a good solution designer or architect. When the business analyst establishes expectations, the designer or architect has tremendous barriers to overcome when trying to recommend more efficient alternatives.
Many metadata documents are produced during this stage, but these should not be considered to necessarily be the development blueprint. Essential information needs to get into blueprint form before this phase is completed.
Design Phase
Once the full business need is analyzed and understood, then the design phase can commence. During this phase, the functional requirements of the solution should be produced. (Note we are still not designing the solution architecture here. We are specifying the functionality of a system to meet the business requirements identified during analysis.)
The most typical problem with this phase is that it gets either short-circuited or overly restricted by premature design work in previous phases. During this phase, the main bulk of the software blueprint should be completed.
Architecture Phase
Only after all the design requirements of the solution are understood should the architecture of the proposed system be planned. During this phase, tools are chosen, object models are created, and the prototype screens are refined.
As in every stage, one of the major problems is premature planning. Remember that the analyst is not necessarily a talented software architect. As pointed out in Anti Patterns (Brown, 1998), perhaps one in 50 developers has a talent for architecture. The same is probably true of analysis skill. It would be exceedingly rare indeed to find the rare talented business analyst to also be one of these rare gifted architects.
Development Phase
All previous phases were in preparation for the development phase. This is where the specified product gets produced. Again, overplanning or premature planning is as much a risk as insufficient planning. While the specification should fully specify all details, sufficient latitude should be reserved to allow the developers to craft the best possible solution.
Using the building analogy, the blueprint fully specifies what wall goes where, but it doesn't attempt to tell the construction crew how many nails to use per stud and where to place them. Those low-level decisions are best left up to the craftsmanship of the builders.
Testing Phase
The developers should perform unit testing as each functional unit is implemented. In addition, independent testers should perform system testing at each internal release. Good programmers welcome and value the assistance of those testers with rare talent to help identify problems. Finally, testing by the client should occur frequently in order to minimize the impact of any miscommunication.
The major type of process overlap that can occur here is when an impatient manager sends code to test, or worse, to the client, prematurely.
Notice that the recurring theme in this section is that, as in building a house, each craftsman should be allowed to specialize4 at his particular task, and be allowed to perform that task only after all requisite work has been completed.