This chapter is from the book
This chapter is from the book
This chapter is from the book
Extended Web Services Specifications
Following the broad adoption and use of the basic Web services specifications—SOAP and WSDL—requirements have grown for the addition of extended technologies such as security, transactions, and reliability that are present in existing mission-critical applications. These extended features are sometimes also called qualities of service because they help implement some of the harder IT problems in the SOA environment and make Web services better suited for use in more kinds of SOA-enabled applications.
A class of applications will find the core specifications sufficient, while other applications will be blocked or hampered by the lack of one or more of the features in the extended specifications. For example, companies may not wish to publish their Web services without adequate security or may not wish to accept purchase orders without reliable messaging guarantees.
The core specifications were defined with built-in extensibility points such as SOAP headers in anticipation of the need to add the extended features.
Standardization
Web services specifications progress toward standardization through a variety of ways, including small groups of vendors and formally chartered technical committees. As a general rule of thumb, most specifications are started by a small group of vendors working together and are later submitted to a standards body for wider adoption. Specifications initially created by Microsoft and IBM, together with one or more of their collaborators (these vary by specification, but typically include BEA, Intel, SAP, Tibco, and Verisign), tend to gain the most market traction. Microsoft and IBM are the de facto leaders of the Web services specification movement and have defined or helped to define all the major specifications. Several of the WS-* specifications remain under private control at the time of writing, but we expect them to be submitted to a standards body in the near future.
Standards bodies currently active in Web services include:
-
World Wide Web Consortium (W3C)—Making its initial name on progressing Web standards, notably HTTP, HTML, and XML, the W3C is home to SOAP, WSDL, WS-Choreography, WS-Addressing, WS-Policy, XML Encryption, and XML Signature.
-
Organization for the Advancement of Structured Information Standards (OASIS)—Originally started to promote interoperability across Structured Generic Markup Language (SGML1) implementations, OASIS changed its name in 1998 to reflect its new emphasis on XML. OASIS is currently home to UDDI, WS-Security, WS-BPEL, WS-Composite Application Framework, WS-Notification, WS-Reliability, Web Services Policy Language (part of the Extensible Access Control Markup Language TC), and others such as Web Services for Remote Portlets, Web Services Distributed Management, and Web Services Resource Framework, which are not covered in this book.2
-
Web Services Interoperability (WS-I)—Established in 2002 specifically to help ensure interoperability across Web services implementations, WS-I sponsors several working groups to try to resolve incompatibilities among Web services specifications. WS-I produces specifications called profiles that provide a common interpretation of other specifications and provides testing tools to help Web services vendors ensure conformance to WS-I specifications.
-
Internet Engineering Task Force (IETF)—The IETF is responsible for defining and maintaining basic Internet specifications such as TCP/IP and SSL/TLS. Their relationship to Web services is indirect in that TCP/IP is the most common communications protocol used for the HTTP transport, and basic IETF security mechanisms are used in Web services. The IETF collaborated with the W3C on XML Signature.
-
Java Community Process (JCP)—Established by Sun to promote the adoption of Java and control its evolution, the JCP is home to several Java Specification Requests (JSRs) that define various Java APIs for Web services, including JAX-RPC for the Java language bindings for SOAP, JAX-B for XML data binding, and Java APIs for WSDL.
-
Object Management Group (OMG)—Initially established to create and promote specifications for the Common Object Request Broker (CORBA), the OMG is home to specifications that define WSDL language mappings to C++ and CORBA to WSDL mappings.
Web services standardization started with the submission of the SOAP 1.1 specification to the W3C in mid-2000. After that, SOAP with Attachments, XKMS, and WSDL were submitted to W3C. At the same time, UDDI was launched in a private consortium and was later submitted to OASIS. Other major specifications submitted to OASIS include WS-Security, WS-BPEL, WS-CAF, and WS-Notification. More recently, WS-Addressing and WS-Policy were submitted to W3C, signaling a potential shift back toward W3C as the home of most of the major specifications.
Historically, OASIS is also the home of the ebXML set of specifications, which overlap to a large extent with the Web services stack. Web Services and ebXML share SOAP, but beyond that, the stacks diverge. ebXML has its own registry and its own orchestration (or choreography) language.
Standardization and Intellectual Property Rights
One of the difficulties with respect to Web services standardization is the fact that no single standards body is clearly in a leadership position. Specifications work is split across W3C, OASIS, WS-I, IETF, and OMG. How does a specification become an adopted standard? If anyone had a magic formula for this, they would be millionaires. In the end, it's market acceptance and adoption that makes the difference, and this means the economic factors become paramount, both in terms of vendor investment and customer procurement. Web services vendors often initiate work on specifications informally in small teams for the sake of rapid progress, publish the specifications themselves, and then submit the specifications to a standards body. Microsoft pioneered this approach with SOAP 1.1, and other vendors including BEA, HP, IBM, IONA, Oracle, SAP, Sun, Tibco, WebMethods, and others have significantly contributed to specifications this way. However, a major question with this approach often arises when it's time to submit the specification to an open standards body. In particular, it's necessary to resolve issues related to intellectual property (IP) rights, including copyrights and patents. It's an important step in a specification's lifetime, even if the standards body doesn't change it very much. Only when a specification is submitted to an open standards body and the IP issues resolved (or at least publicly declared) can that specification truly achieve widespread adoption. Without this step, software vendors not among the initial authoring community have no visibility into specification changes, which could invalidate their investments in products, and they might be faced with potential IP licensing fees, whether royalties or otherwise, that the specification owners might wish to charge.
Specification Composability
As mentioned previously, the SOAP and WSDL specifications are designed to be extended by other specifications. Two or more of the extended specifications can be combined within a single SOAP message header. For example:
<S:Header> <wsse:Security> ... </wsse:Security> <wsrm:Sequence> ... </wsrm:Sequence> </S:Header>
This example illustrates the use of extended headers for security and reliability. The security header typically includes information such as the security token that can be used to ensure the message is from a trusted source. The reliability header typically includes information such as a message ID and sequence number to ensure the message (or set of messages) is reliably received.
Note the separate namespaces used for the security and reliability headers, wsse: and wsrm:, respectively. The headers use different namespaces so that they can be added incrementally to a SOAP message without concern over potential name clashes. Duplicate element and attribute names are not permitted in an XML document (and a SOAP message is an XML document, after all). Namespace prefixes provide a unique qualifier for XML element and attribute names, thus protecting names from duplication. This is one way in which SOAP header extensions work composably with each other.
Adding extended features may or may not require modification to existing Web services—that is, the extended features can be added into the SOAP headers without changing the SOAP body in many cases. But the execution environments and mapping layers may need to change in order to handle the extensions. Certainly at least adding SOAP headers for extended features must be done within the context of knowing whether the execution environment can support them and how; otherwise, the extended headers will not work.
Web service extensions are also added to the responsibility of the SOAP processors in the execution environment. Policy declarations associated with the WSDL contracts can be used during the generation of SOAP messages to determine what should go into the headers to help the execution environment negotiate the appropriate transport protocol or to agree on features such as transaction coordination.
As illustrated in Figure 1-12, each additional extended feature added to the Web service invocation results in additional processing before sending the message or after receiving it. Later chapters provide further detail on each of these extended features. The extended features may also be related to requirements from a business process engine and may need to be supported by the registry.
)
Figure
1-12 Adding extended features to SOAP.
Composability and Complexity
Supposedly, the composability of extended Web services specifications allows their incremental use or "progressive discovery" of new concepts and features. IBM and Microsoft, as the de facto leaders of the Web services specifications development, are obviously keen to avoid making them too complex. A large part—if not the largest part—of the value of Web services derives from their relative simplicity. CORBA, for example, is often criticized for being too complex and too hard to use. Of course, CORBA is easier than what preceded it, but CORBA is relatively complex compared to Web services. The issue with complexity is finding people well enough trained to use the technology productively, and because the highest IT cost is still labor, complexity typically means additional project expense. So for Web services to keep their promises, and to avoid having the whole effort fall apart, IBM and Microsoft want to preserve simplicity even as they start adding complex features to the basic specifications. One argument is that Web services are designed to inherently support composition of new features, meaning existing applications can be extended instead of being changed. However, this is a relatively untested assertion because products that fully implement the extended specifications are not yet available, and it isn't at all clear that the products will implement the extended features in the same way. Furthermore, there is no overall architecture for Web services that defines how the extended features really work with each other. When you receive a complex SOAP message full of headers for security, reliability, and transactions, what do you process first? The security header? The reliability header? The transaction header? No one really knows. If this all proves too complex, people may just go back to plain old XML over HTTP and hand code the extensions.
Metadata Management
Metadata management includes the description information about a Web service necessary to construct a message body (including its data types and structures) and message headers so that a service requester can invoke a service. The provider of the service publishes the metadata so that a requester can discover it and use it to construct messages that can be successfully processed by the provider.
When invoking a service, it's important to understand not only the data types and structures to send but also the additional qualities of service provided (if any), such as security, reliability, or transactions. If one or more of these features are missing from the message, it may prevent successful message processing.
Metadata specifications include:
XML Schema—For message data typing and structuring and expressing policy information.
WSDL—For associating messages and message exchange patterns with service names and network addresses.
WS-Addressing—For including endpoint addressing and reference properties associated with endpoints. Many of the other extended specifications require WS-Addressing support for defining endpoints and reference properties in communication patterns.
WS-Policy—For associating quality of service requirements with a WSDL definition. WS-Policy is a framework that includes policy declarations for various aspects of security, transactions, and reliability.
WS-MetadataExchange—For querying and discovering metadata associated with a Web service, including the ability to fetch a WSDL file and associated WS-Policy definitions.
Service binding is different for an SOA based on Web services compared to an SOA based on J2EE or CORBA, for example. Instead of binding via reference pointers or names, Web services bind using discovery of services, which may be dynamic. If the service requester can understand the WSDL and associated policy files supplied by the provider, SOAP messages can be generated dynamically to execute the provider's service. The various metadata specifications are therefore critical to the correct operation of an SOA based on Web services.
Addressing
Addressing is an important requirement of extended Web services because no directory of Web services endpoint addresses exists on the Web. SOAP messages must include the endpoint address information within the message for all but the simplest MEP. WS-Addressing replaces earlier proposals called WS-Routing and WS-Referral.
Without an addressing solution, when you send a Web service request to a provider, typically the only address the provider has is the return address to the requester, and then only for the duration of the session. If anything goes wrong on the reply, there's no good way to retry it—basically the requester's address can be lost when there's a communication failure. Also there's no good way to specify a different return address than the requester's address. And finally, there's no way to define address schemes or to identify endpoint addresses for complicated message exchange patterns or multi-transport support.
Policy
Policy is necessary for expressing any extended Web services features of a service so that the requester can provide the necessary security, transaction, or reliability context in addition to the data requirements for the messages expressed in the WSDL file.
Policy provides a machine-readable expression of assertions that a service requester must adhere to in order to invoke upon a provider. Does the service require security or support transactions? The latter can be very important when trying to figure out whether or not a long running, complex interaction can involve a transaction, or whether a transaction can span across all the Web services identified for it.
WS-Policy is necessary for achieving interoperability for the extended features because the policy declarations are the only way in which a requester can discover whether a provider requires some or all of the extended features. In the case of security, for example, different providers may support different kinds of tokens, such as X.509 or Kerberos. WS-Security is designed as a kind of open framework that can carry any token type. However, if the token type the provider expects isn't declared, the requester can only guess at what it must be.
When making the decision to invoke the provider's service, it may also be important to discover whether it supports reliability or transactions. You might want to know, for example, whether the provider's service can accept retries if the original submission fails and whether it will let you know when it has successfully received a message. Finally, you may want to know whether or not to send a transaction context to the provider to enroll the provider Web service in the transaction.
Acquiring Metadata
It's possible that a requester will obtain the metadata it needs using WS-MetadataExchange or another similar mechanism that queries the WSDL and associated policy files directly. WS-MetadataExchange uses a feature of WS-Addressing called "actions" to access the metadata published by a service provider. WS-MetadataExchange is designed to provide any and all information about a Web service description—essentially replacing UDDI for most applications.
Developers may or may not use UDDI, despite its existence. It's fair to say that the public UDDI does not provide the metadata management facilities required to support interoperability requirements at the extended specification level and that WS-MetadataExchange may be needed for requesters to ensure they have the information they need to achieve interoperability with providers using extended features.
Security
Security concerns apply at every level of the Web services specification stack and require a variety of mechanisms to guard against the numerous challenges and threats that are a part of distributed computing. The mechanisms may have to be used in combination to guard against a particular threat or combination of threats. In the Web services and SOA world, it's particularly important to evaluate the need for protection at the network layer, the message layer, and for the data in the message.
Basic security protection mechanisms are built around encryption, authentication, and authorization mechanisms and typically include comprehensive logging for problem tracking. The industry has achieved consensus around a single specification framework, WS-Security, although ongoing work is necessary to complete the profiles and additional related specifications.
WS-Security was developed to provide message-level security on an end-to-end basis for Web services messages. Typical HTTP-based security mechanisms, such as SSL, provide only transport-level point-to-point protection. Sometimes additional security may be provided through the use of an alternative transport mapping, such as CORBA's IIOP or WebSphere MQ, but as with the rest of the extended features, the security specifications are written for HTTP as a kind of default or lowest common denominator transport and therefore can be applied to any transport.
WS-Security headers include the ability to carry strong authentication formats such as Kerberos tickets and X.509 certificates and can use XML Encryption and XML Signature technologies for further protecting the message contents. Although a WS-Security authorization profile for the Security Assertion Markup Language (SAML) is being developed, SAML can also be used on its own for exchanging authorization information.
Additional specifications from IBM, Microsoft, Verisign, and others further extend and complement WS-Security, including:
WS-SecurityPolicy—Defines security assertions detailing a Web service's requirements so that the service requester can meet them.
WS-Trust—Defines how to establish overall trust of the security system by acquiring any needed security tokens (such as Kerberos tickets) from trusted sources.
WS-SecureConversation—Defines how to establish and maintain a persistent context for a secure session over which multiple Web service invocations might be sent without requiring expensive authentication each time.
WS-Federation—Defines how to bridge multiple security domains into a federated session so that a Web service only has to be authenticated once to access Web services deployed in multiple security domains.
Because Web services are XML applications, and because XML has security challenges of its own (it is basically human-readable text sent over the Internet), XML-based security technologies are also often important for protecting the XML data before and after it's included in a SOAP message. These technologies include:
XML Encryption—Designed to provide confidentiality, using a variety of supported encryption algorithms, of part or all of an XML document to ensure that the contents of a document cannot be intercepted and read by unauthorized persons.
XML Signature—Designed to provide integrity, using a variety of encryption and signature mechanisms, to ensure that service providers can determine whether or not documents have been altered in transit and that they are received once and only once.
XML Encryption and XML Signature can be used to protect Web services metadata as well as data.
Reliability and Messaging
Messaging includes SOAP and its various message exchange patterns (MEP). The industry has not achieved consensus on a single, unified set of specifications for advanced messaging. However, competing specifications in the categories of reliability and notification work essentially the same way, and so an amalgam of the two is used here for the sake of introduction.
In general, reliable messaging is the mechanism that guarantees that one or more messages were received the appropriate number of times. Reliable messaging specifications include:
WS-Reliability.
WS-ReliableMessaging (from IBM and Microsoft).
Reliable messaging is designed to ensure reliable delivery of SOAP messages over potentially unreliable networks such as the HTTP-based Internet.
Reliable messaging is a protocol for exchanging SOAP messages with guaranteed delivery, no duplicates, and guaranteed message ordering. Reliable messaging works by grouping messages with the same ID, assembling messages into groups based on message number, and ordering them based on sequence number.
Reliable messaging automates recovery from certain transport-level error conditions that the application would otherwise have to deal with on its own. Reliable messaging also supports the concept of bridging two proprietary messaging protocols over the Internet.
Also in the general messaging area are specifications for extended MEPs such as event notification and publish/subscribe, which basically extend the asynchronous messaging capability of Web services. Specifications in this area include:
WS-Eventing.
WS-Notification.
Notification delivers messages through an intermediary often called a message broker or event broker. Subscribers identify the channels or topics for which they wish to receive messages. Publishers send messages to the channels or topics on which subscribers are listening. Notification is a messaging mechanism that can be used to set up broadcast and publish/subscribe messaging.
Transactions
Transactions allow multiple operations, usually on persistent data, to succeed or fail as a unit, such as processing an order or transferring funds. One of the most important aspects of transaction processing technologies is their ability to recover an application to a known state following an operating system or hardware failure. For example, if any failure occurs before a funds transfer operation is completed (that is, both the debit and credit operations), transactions ensure the bank account balances are what they were before the funds transfer operation started.
Many Web services applications may require only the transaction processing capabilities inherent in the underlying execution environment, such as those provided by application servers and databases. Others may require multiple Web service invocations to be grouped into a larger transactional unit, including a transactional context within SOAP headers so that the transaction can be coordinated across multiple execution environments.
Web services transaction specifications extend the concept of the transaction coordinator, adapt the familiar two-phase commit protocol for Web services, and define new extended transaction protocols for more loosely coupled Web services and orchestration flows. Transaction coordinators currently exist in most execution environments, including J2EE, the .NET Framework, and CORBA. Web services specifications define extensions for these (and other) coordinators for compensation-based protocols and long-running coordinator-coordinator protocols that bridge software domains.
Coordination is a general mechanism for determining a consistent, predefined outcome for composite Web service applications. The coordinator model includes two major phases: the acquisition phase in which Web services that are participating in the composite are enrolled with the coordinator for a specific protocol (such as two-phase commit, compensation, or business process) and a second phase in which the coordinator drives the agreed-upon protocol to completion following the end of the execution of the set of services in the composite. When a failure occurs, the coordinator is responsible for driving the recovery protocol (if any).
The specifications in this area include:
-
WS-Transactions family from BEA, IBM, and Microsoft:
-
WS-AtomicTransactions—Defines volatile and durable variations of a standard two-phase commit protocol for short-lived executions.
-
WS-BusinessActivity—Defines variations on the idea of tentative commit and compensation-based undo protocols for longer-lived executions.
-
WS-Coordination—Defines the coordinator for the two pluggable protocols (and their variations).
-
WS-Composite Application Framework (WS-CAF) from OASIS:
-
WS-Context—Defines a standalone context management system for generic context (that is, for non-transaction protocol contexts such as security, device and network IDs, or database and file IDs).
-
WS-CoordinationFramework—Defines a coordinator for the basic context specification and the pluggable transaction protocols in the WS-TransactionManagement specification.
-
WS-TransactionManagement—Defines three transaction protocols for the pluggable coordinator: ACID, long-running actions (compensation), and business process management.
Both sets of specifications are centered on an extended coordinator with pluggable transaction protocols. Both sets of specifications define atomic transactions (i.e., two-phase commit) and compensation-based transactions. WS-CAF breaks context management into a separate specification and adds a third transaction protocol specifically designed for business process management.
Orchestration
Web services can and eventually will be published for most software systems and applications within a given IT environment, and in fact across multiple organizations' IT environments. Rather than have Web services invoke each other using one or more of the message exchange patterns supported by SOAP and WSDL, an orchestration engine can be used to create more complex interaction patterns in long-running business process flows with exception handling, branching, and parallel execution. To accomplish this, the orchestration engine has to preserve context and provide correlation mechanisms across multiple services.
A Web service orchestration may also be published as a Web service, providing an interface that encapsulates a sequence of other Web services. Using the combination of MEPs and orchestration mechanisms, entire application suites can be built up out of Web services at multiple levels of encapsulation, from those that encapsulate a single software module to those that encapsulate a complex flow of other Web services.
The industry has reached a consensus around a single orchestration specification: the OASIS Web Services Business Process Execution Language (WS-BPEL). WS-BPEL assumes that Web services are defined using WSDL and policy assertions that identify any extended features.
Typically, a flow is initiated by the arrival of an XML document, and so the document-oriented Web services style tends to be used for modeling the entry point to a flow. Parts of the document are typically extracted and operated upon by the individual tasks in the flow, such as checking on the inventory availability for each line item from a different supplier, meaning the steps in the flow may be implemented using a combination of request/response and document-oriented Web services.
The WS-BPEL specification differs from other extended specifications in that it defines an executable language compatible with various software systems that drive business process automation. Whereas most other Web services specifications are XML representations of existing distributed computing features and capabilities that extend SOAP headers, orchestration represents the requirement for composing Web services in a declarative manner.
The W3C's Web Services Choreography Definition Language (WS-CDL) is another specification in the general area of orchestration. Choreography is defined as establishing the formal relationship between two or more external trading partners. One difference between WS-CDL and WS-BPEL is that WS-CDL does not require Web services infrastructure at all of the endpoints being integrated.
Strategic Value of Orchestration
Some people say orchestration is where Web services gain their strategic value. Web services have intrinsic value because of the relative ease with which they allow developers to solve interoperability problems across disparate types of software. Web services orchestration can be used in a variety of ways ranging from creating composite services in a declarative (non-programmatic) manner to full-blown business process management. Using Web services orchestration for BPM is clearly more difficult because it requires a deep understanding of an enterprise's business processes. In any case, it is true that the orchestration layer is where everyone expects the solution to be found to the hard problems of data type and structure incompatibilities, semantic data matching, and correlating the results of multiple Web services. It will take a while to prove whether or not these problems can really be solved at the orchestration layer and whether or not automated business process management is something companies really want or need at either the departmental or enterprise level.