Framing the Challenges
It is clear that responding to these pressures requires projects that span multiple business units. This requirement challenges almost every aspect of the way enterprises conduct projects today. In contrast to traditional projects, these new projects require coordinating the work of multiple business units in order to achieve enterprise goals. This coordination is required both in defining the business processes and in the systems development work needed to get those business processes ready for execution. These projects are doing nothing less than modifying the total architecture of the business.
At the heart of this modification lies the determination of who should be doing what in the revised business process: what work should be done by people, and what work should be done by systems. Since both people and systems live in organizational silos, this determination decides what each silo will be responsible for in the revised process. This, in turn, will determine what development work each silo needs to do in order to implement the revised business process.
Therein lies the problem with current project structures. Who, in silo-oriented projects, has the responsibility for revising the total architecture—both business processes and systems? Who defines the needed services? Who has the responsibility for determining the operational and developmental responsibilities for each silo? Who has the authority to make the silos cooperate in this endeavor? How is the overall budget determined and allocated to the silos?
In most enterprises, the silo-based development processes do not contain explicit tasks for determining who should be doing what—either during development or in the revised business process. They have development processes similar to the one shown in Figure 1-2. You either give the requirements to the silo's development team or tell the team to go determine what the requirements are. If there is any figuring out to be done, the silo's development team does this work. If there is work required in other silos, the development team negotiates with the other silos to get that work done. The silo owns the entire project.
Figure 1-2 Silo-Based Client-Server Development
The problem you will encounter with SOA projects is that this streamlined client-server development process doesn't scale. You can't just hand a set of requirements to several development teams, ask each team to figure out what services it ought to be creating, and expect to have an efficient development process that also meets business goals. In fact, with such a fragmented approach, no one is actually accountable for achieving the overall business goals. How can we expect to achieve business goals without such a focus?
An alternative development process looks something like the one shown in Figure 1-3. The project begins with an explicit process charter that provides the vision and focus for the overall effort. This charter sets forth the project goals; establishes the project's cost, schedule, and other constraints; and assigns the key project leadership responsibilities. This process contains an explicit step for determining the services required—the architecture activity. It contains a concession to reality as well: an explicit integration test step. It is impractical to simply turn on a large-scale system with many services for the first time and begin testing. For efficiency, the system needs to be integrated in an organized manner, a few services at a time.
Figure 1-3 SOA Development
The idea of having a clearly defined project charter, an explicit architecture step, and an integration test step is not new by any means. Their inclusion in the development process is a well-established best practice for software development. But if these steps are no longer present in your development process, you must reintroduce them. In addition, your thinking about the scope of the project and its related architecture activity must be extended to encompass business process design as well as system design.
The architecture step is the key to making SOA development work. In this activity, the business process architect determines what services and service orchestrations are needed in the revised business process in order to achieve the business goals. At the same time, the systems architect determines how these services and orchestrations will be provided—and what legacy functionality is required. Together, the architectural responsibilities span both business processes and information systems, both development time and runtime. This step encompasses all of the elements of total architecture.
Closely related to this development process challenge is an organizational challenge (Figure 1-4): Who has overall ownership of a project that spans multiple business units? Who is responsible for the architecture step that determines what each silo should be doing? Where, organizationally, do they report?
Figure 1-4 The SOA Organizational Challenge
This project ownership problem is exacerbated further by a commonplace organizational approach used when introducing new technologies into the enterprise: creating a new IT silo just for the new technology! Unfortunately, this silo, unlike the application silos, does not have a business counterpart (as indicated by the question mark in Figure 1-4). Yet you expect this silo to use the new technology to integrate the other silos. You make it responsible for services, service orchestration, integration, and process management. Yet unless you give it a business side, with authority over the silos involved, you have not created a recipe for success.
Some enterprises are aware of the project ownership issue and the need for overall business guidance in silo-spanning projects. This is evidenced in the structuring of major initiatives like ERP. The business creates a program office that reports to senior management and is responsible for the overall initiative. Successful ERP programs not only address the total architecture issues but also have the authority to ensure the cooperative participation of all the silos involved.
The problem is that more and more projects span multiple silos. In fact, service-oriented architectures make such projects the norm, not the exception. So you need an organizational home for projects that span multiple silos. This home must acknowledge that these silo-spanning projects are deep collaborations between business and IT. SOA projects are not IT projects—they are business projects that have a major IT component.
The final challenge is posed by the notion of reusable services. While you may develop a service in one project, the intent should be to design that service so that it can be reused in subsequent projects (Figure 1-5). Who can provide this cross-project perspective? Who can determine where else the service might be used? Who can look ahead to future projects and anticipate their needs accurately enough to specify a service that will actually satisfy those needs? Who can ensure that future projects will actually use the available services and not reinvent them?
Figure 1-5 Cross-Project SOA Challenges
These diverse challenges are all facets of the enterprise's total architecture. Total architecture spans silos. It spans business and IT. It spans projects, both present and future. In fact, total architecture is the core of the enterprise. It is the structure of organizations, business processes, and systems. It is there, whether you like it or not. So you have a choice. You can choose to turn your back and plod on in ignorance—or you can recognize total architecture for what it is: the very structure of your enterprise. That's what this book is all about.