What about Web-Based Commerce?
Harnessing the Dynamics of an Open Rule Marketplace
Over the past several years, companies have asked us to apply business rule techniques to the relatively new area of Web content management. (I use content here in the sense of what information Web pages hold.) These are typically larger organizations, with many sources of product and pricing content and many Web-based channels (eChannels) through which that information needs to be published. And, as you might guess, these organizations also have large numbers of business rules.
What have we found in these cases? You might think that the problems we diagnosed were all essentially new ones and therefore that all our solutions would be new ones as well. Not unreasonable, but wrong !
The first thing we discovered was a classic case of out-of-control bridges and interfaces (data feeds). I say classic because this problem has been recognized as a fundamental problem in business systems for at least a quarter century [Ross 1978]. The Web environment does put a new face on the problem, but underneath, it is the same as ever.
This problem can be addressed by introducing a universal content repository to keep the total number of bridges and interfaces to a manageable level. For more about this rather simple classic solution, refer to the boxed item, The “Old” Issue of Scalability.
Does the business rule approach also support the new challenges of organizing a highly dynamic environment for Web content management? The answer is a resounding yes. Our experience suggests not only that the business rule approach supports them quite well but also that there might not even be any viable alternative. There are many ways in which this is true, but let me single out three especially important ones.
The “Old” Issue of Scalability
Understanding the Problem
The basic problem with bridges and interfaces (data feeds) is relatively straightforward. Suppose you have a large number of different data sources and a significant number of applications that use them. Typically, these data sources and applications will have accrued one by one in largely unplanned fashion over a number of years. I say years because that is how long it used to take—in Web time, it can now take only months or even weeks.
In such an unplanned environment, each application typically has its own interface to each data source it uses. In the worst case, this means that if there are m number of data sources and n number of applications, you must manage m × n number of interfaces. As the following sample numbers suggest, this total can spiral out of control very quickly as the environment grows over time!
m |
n |
m × n |
---|---|---|
1 |
1 |
1 |
2 |
3 |
6 |
5 |
15 |
75 |
10 |
25 |
250 |
and so on |
and so on |
and so on |
Each interface must be managed. As these sample numbers suggest, the overhead associated with simply managing the interfaces can escalate rapidly. So too can the opportunities for misinterpreting and misusing the source data. Making changes in data definition also becomes increasingly more difficult since such change must be propagated faithfully across the ever-growing number of interfaces. In a word, this approach simply does not scale.
Correcting the Problem
The problem of scalability is one reason why the business rule approach puts such an emphasis on integrating and sharing data. In the ideal case, this solution reduces the number of data sources to m = 1. Now, the growth factor is simply additive. Each time you add a new application, the number of interfaces is simply n + 1, rather than m × n.
Unfortunately, in the real world (yes, that does include the Web), things are usually not that simple, and m cannot be reduced to the minimum m = 1. A common solution is to provide an intermediate staging area—a database or repository into which the data from the m sources is imported and consolidated.
For m data sources, this approach means creating m import interfaces. An export interface is still also needed for each of the n applications. (Now these n export interfaces are from the staging area rather than from the original data sources.) Consequently, the total number of interfaces to be managed is m + n, instead of m × n. Each new data source or application means simply m + n + 1 interfaces to manage. As the following sample numbers indicate, this becomes more and more significant as the number of data sources grows. In other words, this is an approach that does scale.
m |
n |
m × n |
m + n |
---|---|---|---|
1 |
1 |
1 |
2 |
2 |
3 |
6 |
5 |
5 |
15 |
75 |
20 |
10 |
25 |
250 |
35 |
and so on |
and so on |
and so on |
and so on |
The Universal Content Repository
How does this discussion apply to the Web environment? Many companies are beginning to realize that they have a problem with content management. Often, there are a significant number of sources for this content (m) and an ever-expanding number of Web applications (n) that use it. In Web time, the m × n factor can spiral out of control almost before you know it.
The remedy, of course, is to create an intermediate staging area, which can be called a universal content repository. As before, this brings the m × n interface total back to the more manageable m + n case. This is simply an old solution applied to a new problem, but as they say, sometimes the more things change, the more they stay the same!
Developing the Vocabulary
The Web environment brings to the business many new concepts and, in many cases, strange new terms to go along with them (for example, extranets, eMarketplaces, eMarketMakers, affinity sites, commerce engines, and so on). The business rule approach places strong emphasis on defining terms and organizing concepts up front and on doing so in a business-driven manner. This emphasis on vocabulary and foundation business knowledge is just the thing to make sense of the muddle—and to help you do it before any coding starts.
Building for Change
In several of our recent projects, the focus was on building highly tailored eCatalogs that target individual eChannels. Often, these eCatalogs are specific to individual customers and/or promotion efforts. The number of possible variations in composition, pricing, frequency of distribution, and so on (not to mention all the exceptions and restrictions on them, legal and otherwise) is staggering. Furthermore, these variations must be adjustable in close to real time to keep pace with ever-changing business factors.
In the Web-based commerce arena, any architecture that cannot support such real-time adaptability is a nonstarter. What is the optimal approach for making an ever-changing array of selection options accessible and relatively easy to change? This is precisely the area where business rules and business logic technology excel.
Harnessing Marketplace Forces
Although a rule-based approach is essential for dynamic content management, it is not in and of itself sufficient. With so many rules and such rapid change in them, it is almost inconceivable that they can be defined effectively by a central group, no matter how highly qualified that group might be.
Having a central group manage all the rules mimics the top-down control practices of command economies (for example, communism). That approach is inherently flawed at real-world levels of complexity. In macroeconomics, the solution is open marketplaces in which thousands (or hundreds of thousands) of individual consumer choices constantly fine-tune the balance between supply and demand.
A corresponding approach must be adopted for the rules governing use of the universal content repository, covering all the choices to be made about content selection, organization, and delivery frequency for the eCatalogs. These rules must be pushed out to all the individual consumers of the content, that is, to all the individual business users responsible for the particular eChannels.
The term we use for business-side, rule-based specification of any kind of service agreement is eDeal. The particular manifestation of eDeals appropriate for Web content management is eSubscription. Each eSubscription establishes the parameters of a finely tuned pipeline (that is, eCatalog) of content for a particular highly focused Web commerce channel. The eSubscription is set up by the business staff closest to that particular business activity—the people in the best position to determine the optimal tactics and tradeoffs for each particular case.
What about the central group responsible for managing the universal content repository under this approach? To facilitate specifying eSubscriptions, this group would probably define generalized rule templates that reduce much of the business user activity to point-and-click selection of appropriate parameters.
The most important thing to remember about the central group, however, is that it acts as neither creator (supplier) nor consumer of content. Instead, it supports the inner workings of the content marketplace. Working in much the same manner as the support staff for stock or commodity exchanges, the central group's basic role is as enabler of intrabusiness content exchange and as enforcer for the basic corporate rules of fair trade.