The rapid-fire guide to managing high-intensity software projects.
Today's high-intensity, Internet-time projects: they're more than yesterday's management techniques can handle. To succeed, you need to understand what's different, where the pitfalls are, what works, and what doesn't. You need Managing High-Intensity Internet Projects. Legendary IT leader Ed Yourdon delivers instant, practical solutions for key challenges associated with Internet development. You'll discover how to:
Whether you're building B2B or B2C, infrastructure or mobile applications, Managing High-Intensity Internet Projects is your secret weapon-everything you need to deliver outstanding results on Internet time!
Users and managers are becoming more demanding. Many Internet-based projects require BPR to succeed. Peopleware issues are often exacerbated. The pace of business demands faster implementation. Internet-based projects are often exposed to much greater risks than before. New technologies are emerging faster. Conclusion.
Identifying the key players. Determining the basic nature of the project. Managing project definition: What does “success” mean? Estimating techniques. Tools for assisting the estimation process. Tradeoffs among schedule, budget, staff, and quality. What to do when rational negotiations are impossible. Conclusion.
Introduction. Processes, core processes, and process interfaces. The role of IT in a BPR project. Critical success factors in BPR. A BPR management plan. Conclusion.
Developing a business strategy. The impact of the Internet on business strategy. Basic types of business strategy. Implementing the business strategy. Conclusion.
Introduction. Heavy processes. Light/Agile processes. A recommended light process. Conclusion.
Introduction. The importance of requirements. Eliciting requirements from the user. Documenting requirements. Managing the requirements. Conclusion.
Introduction. Design issues. Coding issues. Conclusion.
Introduction. Scheduling the testing activity. The testing process. Categories of testing for Internet-related systems. Criteria for completion. Conclusion.
Introduction. Managing the team's time. Project reviews, walkthroughs, and inspections. Defect tracking against quality targets. The “daily build” concept. Conclusion.
Introduction. Hiring and staffing issues. Loyalty, commitment, motivation and rewards. Team-building issues. Workplace conditions for high-intensity Internet projects. Conclusion.
The minimal toolset. Tools and process. Risks of choosing new tools. Conclusion.
Risk! Risk anything! Care no more for the opinion of others, for those voices. Do the hardest thing on earth for you. Act for yourself. Face the truth.
Katherine Mansfield, in The Journal of Katherine Mansfield, 1927.
The first question that people ask me about managing high-intensity Internet projects is: "What's different?" The very nature of the question implies that IT professionals and managers assume that something must be different about managing such projects ? but at the same time, they suspect that many of the fundamentals of project management must still exist. Thus, there is often a sigh of relief when I tell an audience of IT managers that nothing is different about managing high-intensity Internet projects, and an air of excitement and dread when I tell them that everything is different.
The objective of this book is to focus on what's different, though I'll also take the opportunity to remind you occasionally about the fundamental principles of project management so that you dare not forget in the midst of your excitement over exotic new Internet-based technologies. As a reader, I'll assume that you're familiar with the basics of organizing, scheduling, estimating, recruiting, and supervising the myriad details associated with an IT development project. Thus, I won't try to explain what a Gantt chart is used for, or why someone would believe that a work breakdown structure is a critical tool for managing projects, nor will I give you a blow-by-blow description of the typical life-cycle activities of analysis, design, coding, and testing. But I'll also assume that you're unaware of, or unprepared for, some of the unique pressures and difficulties associated with the kind of high-intensity Internet projects that are being launched in thousands of companies around the world today. Specifically, here's what's different about such projects:
Each of these points is discussed in more detail below, and each point forms the basis of a subsequent chapter.
During the last few years, many pundits, university professors, consultants, and business magazines have exhorted management to "reinvent" their companies to take advantage of Internet-based supply chains, global e-commerce possibilities, and internal efficiencies. In many cases, there was a great deal of hype associated with these exhortationsbut it's important to realize that many business managers took them seriously. In some cases, they believed that their very existence was threatened; for instance, a traditional bookseller might well have felt threatened by Amazon, and a traditional newspaper publisher might well have felt threatened by the existence of Internet-based news. And wherever the Internet offered the opportunity to access new markets, new customers, new products, or new services, there was enormous pressure to be first to market and thus exploit the opportunities before additional competitors arrived on the scene.
As this book was being written in mid-2001, the hype associated with the Internet had begun to fade and was being replaced in some companies with jaded cynicism. Perhaps it's because the Internet has now become a more familiar technology; in both the business world and society. As of early 2001, for example, surveys indicated that approximately 50% of U.S. households had access to the Internet, and the figure was approximately the same in many other industrialized nations around the world. Perhaps the hype has faded because of the collapse of the "dot-com" industry in the 2000-2001 period; no longer do we necessarily expect a startup company to have a greater market capitalization than General Motors just because its name is XYZ.com. In any case, many companies had scaled back or postponed their e-business plans by the early part of 2001, and there may be a brief respite of a few years before we see a return to the frenetic pace of the late 1990s.
But the hype has not disappeared completely, and the demands of the business community for Internet-based systems have not disappeared, either. Yes, there are a few simple Internet-based projects, and a few situations where business managers casually say to the IT department, "Whenever you folks have some spare time, would you mind creating a little Web site for us?" Indeed, as Cohen, Grochow, and Raiffa point out (see Figure 1.1), the majority of Internet/Web-based applications in the late 1990s were nothing more than simple, static Web pages.
It's more common today to see organizations regarding the Internet as a "disruptive technology" that creates a global, 24-by-7 business environment for their products and services, instead of the Monday-to-Friday, nine-to-five business environment in which they interacted primarily with pre-existing customers in their local marketplace. Using the Internet for both internal operational purposes and also for external customer-facing activities is no longer optional for any but the smallest of businesses, and for all but the smallest business, the Internet forces business managers to reconsider such fundamental questions as:
Many companies are just now beginning to realize that the Internet is providing their customers with far more power than ever before; not only can customers compare prices and service offerings from many competitors more easily than before, they can also interact with other customers to share experiences and even aggregate their individual orders to obtain better prices. Until the Marketing/PR Department of XYZ Corp. discovers that there is an active Web site called "WeHateXYZ.com"visited daily by thousands of irate XYZ customers who want to share their experiences with one anotherthey don't really understand the familiar phrase: "The Internet changes the traditional relationship between suppliers and customers."
So, what does all of this have to do with the issue of managing Internet-based projects? Aside from the general observation that these projects tend to be more intense and high-pressured than the traditional projects we may have worked on previously, it also implies that there will be more emphasis on the politics associated with such a project. As we'll discuss in Chapter 2, this means that the project manager has to be more concerned than ever with questions such as:
The official and visible aspects of project success may involve an aggressive deadline, a tight budget, and the ability to accomplish the work with a limited staff; but the invisible criteria for success are likely to involve the re-engineering, or reinvention, of business processes. Hence, there is likely to be a great deal of emphasis on negotiations concerning deadlines, schedules, budgets, and project staffing. These are familiar topics for most project managers, but the negotiations often turn out to be more intense and demanding for Internet-based projects. More importantly, though, there is likely to be more emphasis than ever on BPR; we'll discuss this in more detail below.
As noted earlier, the IT industry went through another technology upheaval in the early 1990s, when client/server technology first appeared on the scene and companies had the chance to replace mainframe-based, "dumb terminal" systems with decentralized GUI-based applications. While there was a great deal of fascination with the hardware and software technologies associated with client/server systems, it quickly became apparent that real business improvements would only be accomplished if the business processes were re-engineered when the new technologies were deployed. This led to the popularization of a new buzzword"BPR," for business process re-engineeringand it also introduced some less popular buzzwords like "downsizing" and "right-sizing" as companies attempted to use the new technologies to eliminate clerks, administrative personnel, and layers of middle management.
Curiously, many of the lessons and experiences associated with this era have already faded from memory; indeed, when I use the term "BPR" in seminars and presentations today, I find that many of the current generation of Internet-savvy developers and managers have no idea what I'm talking about. As a result, they're also unfamiliar with the work of such management gurus as Michael Hammer, who helped evangelize the BPR movement in the early 1990s with his seminal book, Reengineering the Corporation. And perhaps more important, they're unaware of the sobering assessment made by Hammer toward the end of the 1990s: 80% of the BPR initiatives had failed to achieve their objectives. The reason for the failure, in most cases, had little or nothing to do with the glamorous client/server technology; in most cases, it had to do with politics, culture, and the difficulties of organizational change.
In the extreme case, both the client/server projects of the last decade and the Internet-based projects of the current decade were sometimes launched with no business model for identifying benefits, revenues, customers, or any rationale for successother than the Field of Dreams theory that "If we build it, they will come." This is the common explanation for the failure of so many dot-com enterprises, and it also explains the failure of many internal e-business projects that companies have launched.
But not all BPR projects of the client/server era were spectacular failures, and not all of the ill-considered Internet projects will be as visibly catastrophic as the collapse of Pets.com or wine.com. Many of the less visible BPR failures of the 1990s turned out to be, in Hammer's words, "re-paving old cowpaths"that is, using new technology to carry out the same old (inefficient) business processes slightly faster and cheaper with new hardware and software. Alas, the same thing is happening today with the latest technology: the Internet. For example, one government agency decided that the Internet would be a wonderful technology for replacing its paper-based, hard-copy employee directory, which was updated on an annual basis by the Human Resource Department, and was thus obsolete for most of the year. The new system had a slick, colorful Web interface and could be accessed by employees anywhere in the government agency; unfortunately, the Human Resource Department insisted that only it could update employee-related information ? which it continued to do on an annual basis. Thus, the new system saved a lot of paper and provided a user-friendly means of displaying information, but the information was still fundamentally obsolete because the organization refused to change its business processes.
Managers of high-intensity Internet projects must realize that part of their role is to be the BPR champion or facilitator, to ensure that the underlying business processes are re-engineered when the new technology is deployed. But there is a big risk here: Most project managers lack the authority or political power to impose BPR on the business departments that are clamoring for new Internet technology. There is no magic solution to this problem, and no guarantee for success. We will discuss the BPR issues associated with Internet projects in considerable detail in Chapter 3.
For most of the past decade, if not longer, IT organizations have been faced with a shortage of technical skills. In some cases, this has led to deferral and postponement of projects; in other cases, it has led to understaffed projects, with three developers trying to accomplish what normally would have been assigned to a team of five. But during the hey-day of the dot-com era, it also led to inflated salaries, outrageous employee benefits (e.g., a new car as a signing bonus), and unrealistic expectations on the part of freshly minted college graduates who expected stock options and compensation packages on a par with Bill Gates.
The dot-com frenzy of the late 1990s also led to an increase in mercenary behavior, with talented developers jumping from one job to another without any concern for company loyalty. On the other hand, this is a phenomenon that can also be traced back to the client/server era, in which IT departments made cold-blooded decisions to replace highly paid COBOL programmers with younger, cheaper Visual Basic programmers. The Dilbert-style cynicism that pervaded many IT organizations during this period has continued to this day, so that both employers and employees have forsaken any pretext of loyalty and long-term commitment; the result is that managers of high-intensity Internet project cannot necessarily assume that their team will remain intact for more than a few months.
As this book was being written in spring 2001, the dot-com collapse and associated economic slowdown had made many IT professionals more cautious than before and less willing to accept job offers consisting of low salaries and stock options of dubious value. At the same time, it made some of them more cautious about abandoning stable jobs in boring brick-and-mortar companies to work on glamorous Internet projects in new startup companies; as a result, project managers may have found a little more personnel stability than in recent years. On the other hand, the same economic slowdown led to cuts in training budgets and various other belt-tightening decisions that may have exacerbated a somewhat tenuous peopleware situation in some IT organizations. This can be a particularly serious problem in companies that expect their IT developers to work long overtime hours on high-pressure Internet projects; it can be extremely demoralizing to learn that the Accounting Department won't approve a $5 expenditure for a dinner-in-the-office slice of pizza and can of soda at the end of a 14-hour workday.
It's important for the manager to be aware of these problems, especially because many of today's Internet projects also involve team members who have no formal IT background. It's enough of a shock to hear the members of one's project team say, "Testing? What's that? Doesn't this stuff just work when we build it?", but it can be a deeper shock to realize that the artists, musicians, programmers, and subject matter specialists have an entirely different set of assumptions, expectations, and attitudes about their role in the workplace. Just as with BPR, there is no instant solution to the peopleware problems associated with high-intensity Internet projects. We'll discuss these issues in more detail in Chapter 11.
For someone entering the IT industry today, it's amazing to think that once upon a time, application development projects had 7-year schedules. Well, maybe not every project lasted that long, but it was extremely common throughout the 1970s, and even the 1980s, to see projects that took 3-5 years from initiation to full-scale deployment. One of the interesting consequences of the client/server revolution of the early 1990s was that the 7-year projects got turned into 7-month projects, on almost an overnight basis. Again, not every client/server project had this characteristic, but it was extremely common to see a mandate from senior management that no IT development project would be allowed to take longer than an annual budget cycle, from beginning to end.
For those who had been accustomed to the leisurely pace of a 7-year project, this was quite an adjustment. Not only did we see the introduction of programming tools and development environments that supported faster development cycles (e.g., the replacement of batch-oriented COBOL development with interactive tools like Delphi, PowerBuilder, and Visual Basic), but we began to see IT organizations replacing the old-fashioned, time-consuming waterfall development process with various forms of rapid application development (RAD) processes. But a client/server project lasting 6 months, 9 months, or 12 months still had some time available for traditional activities like documenting requirements, developing design models, and conducting code reviews.
Now, for better or worse, we're living in "Internet time," and our Internet development projects have accelerated what was already a rather fast-paced schedule of activities. The old-fashioned projects that once took 7 years and got compressed down to 7 months in the client/server era are now being squeezed into 7 weeks. This is not an exaggeration. One of the large software development firms that I'm associated with regularly takes on client assignments to build Internet-based systems that are described by the client as "mission-critical," and for which the development schedule is 4 weeks long. In some cases, the accelerated schedule is mandated by competitive pressures or "market window" opportunities; in other cases, it's mandated by emotional demands from senior managers who seem to believe that outrageous development schedules can be achieved by executive fiat. And to some extent, all of this reflects the steadily accelerating pace of everything in society today: We're all living in Internet time, and our IT development projects are just a reflection of that fact.
But the consequences for the IT project manager are serious: The imposition of an "impossible" deadline and a breakneck schedule tempt the developers into abandoning all forms of documentation, planning, thinking, analyzing, and formal software development processes and methods. This was bad enough in the early 1990s, when most of the developers at least knew what kinds of processes and methods they were abandoning. But today's Internet systems are often being developed by recent university graduates whose entire exposure to software engineering consists of an introductory course in Java programming. This may seem like an unnecessarily harsh criticism if the university graduate has a degree in software engineering or computer science; but even in this case, we're talking about people who have only worked on solo projects (i.e., their own programming assignments) lasting somewhere between a few hours and a semester.
Refer back to Figure 1.1 and it's easy to see why the early Internet systems did not require a great deal of formal methodological detail: We were building simple Web applications that merely displayed HTML-based pages of information. But many organizations are now beginning to tackle Internet projects that are large enough and complex enough that at least some of the old-fashioned ideas of planning, analyzing, and implementing software development processes are not only relevant, but downright necessary.
For the relatively small, fast Internet development projects that still represent the majority of today's projects, the emerging trend is to use so-called "agile" or "light" software development methodologies to strike a compromise between the bureaucratic paralysis associated with the more formal, rigorous methodologies of the past and the utter anarchy that characterized so many of the Internet development projects of the late 1990s and early 2000s. Striking such a balance is one of the toughest jobs for the Internet project manager; we'll devote an entire chapter to this topic later in the book.
Just getting out of bed in the morning is a risky proposition; everything we do in life has associated risks. And certainly, this has been true of all IT development projects since the beginning of the computer era; there are numerous surveys, articles, and books that document the relatively low percentage of such projects that have been finished on time, within budget, and with users who feel that their requirements have been properly understood and implemented. For Internet systems, as with most other types of computer systems, there are the potential problems of performance (poor response time), reliability (frequent crashes or incorrect results), security (vulnerability to hacking or illegal entry), privacy (disclosure of confidential information to unauthorized persons), and so forth.
But there are at least two reasons why Internet-based systems often have more severe risks than older, traditional systems. The first reason is obvious: By its nature, an Internet-based system is exposed to the entire global population. A teenage hacker in Moscow or Minneapolis or Mozambique may decide to invest his or her considerable skills to hack an organization's Internet system to: (a) shut it down completely, (b) steal money or valuable information, (c) prevent anyone else from accessing the system through "denial-of-service" attacks, (d) post embarrassing messages or pornographic pictures on the system, or (e) all of the above. And second, even without attempting such destructive behavior, it may turn out that several thousand (or million) such teenage hackers decide to visit a system at the same time, thus overloading it to the point where nobody can use it at all. While this might have been theoretically possible with the client/server and online systems of the 1990s and 1980s, it was rarely a practical issue simply because the developers and operators of such systems had more direct control over the community of actual and potential users.
A related point is worth mentioning: Many of today's systems have relatively short lifetimes from an operational perspective. Consider, for example, the Web site and related Internet applications associated with the Olympics, which lasts for approximately two weeks, once every four years; for a more extreme example, consider the Internet system developed to support the Hollywood Oscar awards, for which the ceremonies last approximately four hours. If such a system fails, or is overloaded, or is successfully attacked by vandals or hackers, there may not be enough time to recover from the problem before the associated event is finished. Obviously, not all Internet systems fall into this category, but it reminds us that one of the important ramifications of Internet time is that we no longer have a leisurely period during which to analyze and repair operational failures, and then restore the system in a calm, orderly fashion.
All of this puts more emphasis than ever on careful planning and testing; we may have been able to get away without them in the early Internet projects, but not any longer. Perhaps more important, today's Internet projects are different from the more traditional projects that many of us have managed in the past because they require a greater emphasis on risk management as a significant part of project planning and project management. We'll discuss this topic in more detail in Chapter 10.
Finally, Internet projects are likely to be different than other projects that have taken place within an IT organization simply because they represent the leading edge of technology within the organization. At the same time, the subject matter or business domain associated with the project may either be old or new, which gives us the four distinct possibilities illustrated in Figure 1.2.
In the lower left quadrant of Figure 1.2, we see a combination of old technology and old business/subject matter practices-for example, the implementation of an old-fashioned batch mainframe payroll system in COBOL. Presumably, such projects are still taking place in various parts of the world, and presumably they continue to provide value, though sometimes they are simply re-implementations of an even older technology that has collapsed from old age.
Of more interest are the situations represented in the top left quadrant and bottom right quadrant, that is, the combination of new and old. As discussed earlier, the use of new technology (such as the Internet) to automate old business practices is simply the re-paving of cowpaths, and we are almost certain to see some pressures when moving a project into the top right quadrant so that new technology can be employed to help an organization conduct its business in an entirely new way.
Similarly, it's possible to imagine a radically new and different set of business processes being carried out with old-fashioned technology; sometimes a simple spreadsheet or flat-file database may be sufficient technology to support an innovative business process. But chances are that we'll feel pressure here, too: If the new business is effective with old technology, there's a reasonable chance that it will be even more effective with new technology.
In any case, today's Internet project managers are likely to find themselves coping with new hardware, new architectures, new programming languages, new development environments, new databases, and new standards. Of course, this has been true for each of the waves of technology that have swept through the IT industry during the past 50 years, beginning with the vacuum-tube ENIAC computer. A manager going through his or her first such transition may find the experience daunting, but someone who has been in the field for 10 years or more is likely to recognize that today's new technologies are, in Yogi Berra's terms, "Deja vu all over again." Of course, the new technologies-whether they are today's Internet-related technologies or the next-generation technologies of the middle or later part of this decadecan represent a serious problem if the IT staff doesn't have adequate expertise in the technologies.
This usually leads to a rather chaotic period of a few years, in which the organization outsources some of its high-tech requirements, recruits new employees to acquire the needed skills, trains some of its existing staff to upgrade their skills, and tries to do the best it can with whatever superficial skills it happens to have. Similarly, there is often a rather chaotic period during which new versions of tools, languages, and development environments are introduced by the technology vendorsa period in which the technology doesn't work reliably, or isn't supported adequately by the vendors. Whether it's IBM, Microsoft, Sun, or any one of a dozen other high-profile vendors, such criticisms and accusations were common during the early stages of the Internet era. And as long as these vendors continue to introduce dramatically new technologies, it's likely to continue being a problem.
We'll discuss the role of tools and technologies in the development of Internet-based systems later in this book, in Chapter 12, but my experience has been that new technologies are not the fundamental causes of project failures. In rare cases, the entire project-and the associated business justification for the project-may depend on advanced technology that doesn't work at all, or doesn't get delivered when the vendor promised. But more often, the technical problems are little more than the "straw that breaks the camel's back," providing a handy scapegoat to cover up the more fundamental problems of poor planning, poor management, or poor development processes.
While the specific tools, products, and technologies associated with the Internet may be new and different for many IT managers, the general issues associated with new technologies have been discussed for many years. As Geoffrey Moore explained in his popular book, Crossing the Chasm in the early 1990s, new technologies are adopted by five distinct segments of the marketplace, beginning with "innovators" and ending with "laggards" (refer to Figure 1.3). What Moore did not explicitly note in his book, and what many IT managers have failed to recognize, is that the period of time between the innovators and the laggards, for the adoption of IT technologies, is usually between 14 and 20 years. Thus, the adoption of Java and many of today's familiar Web-based technologies reached the "early adopter" stage by early 2000, but it's likely to be closer to 2010 before the laggards make a commitment.
In any case, the issue of technology adoption is one that Internet project managers must be aware of, for the projects they are managing represent the organization's effort to introduce and deploy the latest IT technologies. Thus, it's important for project managers to understand whether their organizations have traditionally been innovators, laggards, or somewhere in between with respect to new computer technologies; for example, were they among the first to introduce Windows 2000, Linux, object-oriented programming, client/server technology, CASE (Computer-Aided Software Engineering) technology, relational databases, structured programming, and a long list of other technologies? Or were they content to let someone else be the pioneer, and settle instead for the second-prize title of early adopter? Or were they part of the mainstream community, or perhaps even a laggard? Chances are that whatever organizational behavior was exhibited with previous technologies will be repeated again with the Internet, unless the company has a new leader who is determined to make the company either more adventurous or more conservative than it has been in the past.
In many cases, it's also important to understand whether one's customers, vendors, suppliers, and partners are innovators, early adopters, mainstream participants, or laggards. Indeed, one of the main reasons for the failure of so many dot-com companies was not that they were unsuccessful in deploying sophisticated Internet technology, but rather that the marketplace on which they depended was more conservative than had been estimated. Of course, it's possible to build an Internet-based system whose technologies are used solely by internal employees, but for a larger and larger percentage of the Internet systems being built today, the very raison d'etre of the system is to facilitate more effective communications with individuals and organizations outside the corporate boundary. And if those external entities are not yet using the latest wireless/Web/Internet technologies, it may not matter whether you were successful at implementing such technologies internally.
So, what's different about managing high-intensity Internet projects? Well, you still have customers and users who have to be satisfied. You still have schedules and deadlines that someone expects you to meet. You still have a budget that you're not supposed to exceed. You still have a team of developers whose work is abstract and intangible, but whose progress and deliverables still have to be managed, assessed, and controlled. You still have to plan and organize, and you still have to cope with politics. Plus, you still have to deal with unexpected surprises in every facet of the project, from technological glitches to perverse users to headstrong developers. In those senses, nothing has changed. A project is still a project, even if it is a high-intensity Internet project.
On the other hand, much of the familiar terrain of project management has changed so drastically that it's a whole new ballgame. A project operating on a 4-week schedule is qualitatively different than a project operating on a 4-month schedule; as a colleague of mine observed, if it takes you a week to realize that your programmers haven't produced any working code, you've lost 25% of the entire project schedule. Similarly, an Internet-based project whose launch is broadcast on the national TV news (as was the case for another colleague of mine) is operating in an entirely different world than an old-fashioned project that had the luxury of being rolled out in a carefully controlled fashion to one branch office after another. The differences, as we've seen in this introductory chapter, can be both profound and daunting.
But that doesn't mean that we should give up. Indeed, one of the intriguing lessons from the dot-com era (which implies that it's now part of past history, much like the Age of the Dinosaurs) is that very few, if any, projects failed because of mistakes in traditional project management. They may have run out of money, and they may have discovered that there weren't enough customers willing to pay enough money to generate a profit. But most of the systems did get built, and they did have real customers who got real products and services from Internet systems. So, with a little bit of luck, and a lot of hard workand with the guidelines, warnings, and insights that I'll provide in the subsequent chapters of this bookit's possible to succeed.
So, while you may be sobered by some of the problems mentioned in this chapter, you shouldn't give up. Indeed, by the time you finish this book, you should be ready for almost anything ... except, perhaps, whatever comes after the Internet!