Water, Water everywhere, But Not a Drop to Drink.
—Samuel Taylor Coleridge
Ryme of the Ancient Mariner
The belief that using commercial components will simplify the design and implementation of systems is widely held, but is, unfortunately, belief in a compelling myth. Imagine you are the architect or chief engineer for a project that is developing a large, mission-critical information system. For whatever reason, it has been decided that the only practical means to this end—although a very promising means—is to build the system from commercially-available components. Before the project makes substantial progress, though, it is confronted by the following obstacles:
Many crucial design decisions revolve around component interfaces that are deficient in several respects, but are not subject to negotiation. Yet these same interfaces are often unexpectedly modified by their vendor, simultaneously invalidating previous design workarounds and introducing new integration difficulties.
The components have been designed to be easy to integrate with some vendors' components but difficult to integrate with others. Unfortunately, the best components reside in different vendor camps. You discover that the question of which components to use is more complex than anticipated, with each selection decision contingent on some set of other selection decisions.
The components that have been selected are sufficiently complex that no one on the project knows exactly how they work. When integrated, the component assembly exhibits baffling and incorrect emergent behavior. The vendors have never seen this behavior before, but they are certain the fault lies with another vendor's component.
Does this sound familiar? How did this state of affairs come about, and what can practicing designers and software engineers do about it? Can predictable and repeatable software engineering methods be established in the face of these challenges? We believe the answer is yes, and this book shows you how to do it.
1.1 The Software Component Revolution
Software components are everywhere—or perhaps we should say this about magazine articles, scholarly papers, market forecasts, and (yes) books about software components. Judging by all that has been written, and continues to be written, the software industry is quickly moving towards component-based development. Add to this the avalanche of commercial software methods and tools that support component-based development and the conclusion seems inevitable.
A closer look at the evidence, however, presents a picture that is far less clear. We do not argue against this trend. Instead, we propose that this trend, far from being unified and coherent, is a splintering of many trends, some competing, some reinforcing, and some wholly independent. Organizations interested in adopting component-based development for competitive or other reasons are immediately confronted by a nearly impenetrable and fast growing thicket of technological, methodological, and organizational options.
To state the conclusion first, software components are real and are already profoundly alterning the practice of software engineering. Despite signs of progress, though, the challenges posed by systems that are constructed predominantly from commercial off-the-shelf components have yet to be adequately addressed. Indeed, the more general challenges posed by large-scale reuse of existing components, regardless of their origin, have yet to be addressed. Further stipulating the commercial origin of components merely adds further complexity to these already unaddressed challenges.
To be sure, there are several accepted and well-defined component-based development methods, notably Cheesman and Daniels' UML Components , D'Souza and Wills' Catalysis , and Herzum and Sims' Business Component Factory , each recently published and each a truly excellent contribution in its own right. However, these methods assume that the design task is, predominantly, one of component specification for later implementation rather than one of component selection for later assembly. Only Herzum and Sims recognize this apparent contradiction between the premise of currently-published component-based development methods and the consequence of adopting a component-based paradigm; namely, that applications would be assembled from existing components rather than from custom-developed components. Herzum and Sims address this by describing a reuse process (the business component factory) which will shift, ever so gradually, the emphasis from custom development to reuse of existing components.
But surely this misses two key points. First, design methods that assume the freedom to define component interfaces are fundamentally different from methods that lack this freedom1. Defining a method for the former condition in the hopes of transitioning this same method to the latter condition is doomed to failure. Second, there are many systems being built today whose fundamental design challenge stems from an aggressive use of commercial software components. By commercial components we mean things such as Web browsers and servers, object request brokers, relational database management systems, message-oriented middleware, public key infrastructure, transaction monitors, geographic information systems, seemingly ad infinitum. Software engineers need design methods to deal with this class of component-based system now, not in some indefinite future.
Our book addresses this gap in software engineering methodology. In particular, we define concepts, techniques and processes to address the design challenges uniquely posed by the use of commercial software components. These challenges include but are not limited to:
Critical design decisions must be made in the face of incomplete and often mistaken understanding of the features and behavior of commercial software components. Knowledge gaps are inevitable and are a major source of design risk.
Whatever knowledge is obtained about one commercial software component does not to translate easily to components from different vendors, and all component knowledge tends to degrade quickly as components evolve through new releases.
Competitive pressures in the software marketplace force vendors to innovate and differentiate component features rather than stabilize and standardize them. This results in mismatches that inhibit component integration and inject significant levels of wholly artificial design complexity.
Use of commercial components imposes a predisposition on consumers to accept new releases despite disruptions introduced by changing component features. These disruptions take on a random quality across all phases of development as the number of components used grows.
These challenges all derive from the same root cause: a loss of design control to market forces. Perhaps the reason why no methods have yet been developed to address these challenges is that we have been trained to think of market forces as beyond the scope of software engineering methods. But it is a poor engineering method that fails to accommodate practical reality. These market forces play as much a part in software engineering as friction plays in mechanical engineering.
Before delving into details we must first set some context. We describe the class of design problem addressed by this book, especially with respect to different conceptions of software component and component-based development. Then, we outline our assumptions about the software engineering methods and processes that we expect to already be in place. Last, we deal with terminology.