Cargo Cult Software Engineering
In the South Seas there is a cargo cult of people. During the war they saw airplanes with lots of good materials, and they want the same thing to happen now. So they've arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head for headphones and bars of bamboo sticking out like antennas—he's the controller—and they wait for the airplanes to land. They're doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn't work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they're missing something essential, because the planes don't land.
I find it useful to draw a contrast between two different organizational development styles: “process-oriented” and “commitment-oriented” development. Process-oriented development achieves its effectiveness through skillful planning, use of carefully defined processes, efficient use of available time, and skillful application of software engineering best practices. This style of development succeeds because the organization that uses it is constantly improving. Even if its early attempts are ineffective, steady attention to process means each successive attempt will work better than the previous attempt.
Commitment-oriented development goes by several names including “hero-oriented development” and “individual empowerment.” Commitment-oriented organizations are characterized by hiring the best possible people, asking them for total commitment to their projects, empowering them with nearly complete autonomy, motivating them to an extreme degree, and then seeing that they work 60, 80, or 100 hours a week until the project is finished. Commitment-oriented development derives its potency from its tremendous motivational ability—study after study has found that individual motivation is by far the largest single contributor to productivity.2 Developers make voluntary, personal commitments to the projects they work on, and they often go to extraordinary lengths to make their projects succeed.
When used knowledgeably, either development style can produce high-quality software economically and quickly. But both development styles have pathological look-alikes that don't work nearly as well, and that can be difficult to distinguish from the genuine articles.
The process-imposter organization bases its practices on a slavish devotion to process for process's sake. These organizations look at process-oriented organizations such as NASA's Software Engineering Laboratory and IBM's former Federal Systems Division. They observe that those organizations generate lots of documents and hold frequent meetings. They conclude that if they generate an equivalent number of documents and hold a comparable number of meetings they will be similarly successful. If they generate more documentation and hold more meetings, they will be even more successful! But they don't understand that the documentation and the meetings are not responsible for the success; they are the side effects of a few specific effective processes. We call these organizations bureaucratic because they put the form of software processes above the substance. Their misuse of process is demotivating, which hurts productivity. And they're not very enjoyable to work for.
The commitment-imposter organization focuses primarily on motivating people to work long hours. These organizations look at successful companies like Microsoft, observe that they generate very little documentation, offer stock options to their employees, and then require them to work mountains of overtime. They conclude that if they, too, minimize documentation, offer stock options, and require extensive overtime, they will be successful. The less documentation and the more overtime, the better! But these organizations miss the fact that Microsoft and other successful commitment-oriented companies don't require overtime. They hire people who love to create software. They team these people with other people who love to create software just as much as they do. They provide lavish organizational support and rewards for creating software. And then they turn them loose. The natural outcome is that software developers and managers choose to work long hours voluntarily. Imposter organizations confuse the effect (long hours) with the cause (high motivation). We call the imposter organizations sweatshops because they emphasize working hard rather than working smart, and they tend to be chaotic and ineffective. They're not very enjoyable to work for either.