Fusion: Eventing with SOA Part 2 - Eventing using Synapse and WSO2 ESB
Table of Contents
Introduction to Eventing
Before going into details of eventing, it is better to clarify the definition of what an event means. Does an event has a clear cut definition? Actually not. In fact, each person could define an event in his/her own way based on the context of the application of this term. As a simple definition, an event can be a message generated as a response to a change in process or state. The process can be a business or a non-business process. A passenger crosses the road, train arrives at the station, successful payment made through ebay, patient register at hospital, borrow books from library, trade 2000 MSFT shares from NYSE are example process changes that can generate events. Events carry content related to process or state change. Therefore an event generated during a trade process will carry trade info, symbol, quantity, price and side in the event, where as an event generated during a hospital registration will carry patient information.
Eventing works on the asynchronous publish subscribe model but let's keep technical details of eventing to look at later. For now, we will take an everyday example in order to understand eventing better.
If we need to obtain a newspaper subscription, we could go to a distributer and register by paying for the required number of issues. Once we do that, we will continue to get a copies of the newspaper starting with the next immediate publication. When subscription expires, you may either renew the subscription or unsubscribe if we are no longer interested. If two copies are needed we'd make additional subscriptions. If subscription to a different newspaper is needed, we'd then make the subscription with the newspaper name. Subscribers could even call up a dealer to find out details about subscribing. One may subscribe on behalf of a friend/sibling or partner and request newspaper issues be sent to their addresses. Or, perhaps this particular distributer handles only a limited set of newspapers. Then, if you require anything in addition then you may need to call another distributer for subscription. A distributer may receive subscriptions from a single publisher or multiple publishers and may use multiple delivery modes to send newspapers to subscribed users (such as mail, currier etc.) A distributer will keep a registry for recording subscriptions and perform accurate distribution of the newspaper.
Figure 1 - News publisher process
The above example matches precisely with an eventing scenario. We will discuss the terminology of eventing in the next paragraph below.
Eventing contains event sources that generate events. These are the checkpoints that generate events based on state changes. Event distributions handled by an event broker assists a subscription manager in order to manage subscriptions. Event sink(s), or in other words those parties interest in a particular event, need to first subscribe their interest to that event by notifying the subscription manager. Subscription can have a validity time period or indicate as does not expire. An event sink can unsubscribe or renew a subscription at any time. For proper communication between an event source, an event broker, a subscription manager and an event sink, can done using any protocol such as HTTP/s, JMS, mail etc and this communication layer is referred to as the event channel. In essence, we can map the real world example of newspaper subscription and delivery to the eventing terminology as follows:
- user = event subscriber and the event sink
- distributer = event broker
- publisher = event source
- subscription registry = subscription manager
- distribution media = event channel
- newspaper name = filter/topic
Does an event has a standard format? Again, the answer is no. However, there are standard formats that defines a format for an event, which contains a header and a body that carries event data. WSDM (Web Services Distributed Management) WEF (WSDM Event Format) is a example of an event format. Regardless of standards, any message that carries required data/information can be treated as an event. But an event should have some metadata for the subscription manager to identify the event and let event broker dispatch the event based on matching subscribers. Relevant metadata will vary based on the implementation of the subscription manager. As an example, for a topic based subscription manager event should carry the topic name, and for an xpath based subscription manager, the event should format the content to match the xpath expressions.
Event Driven Architecture (EDA) is an architectural pattern build on top of the asynchronous publisher-subscriber integration pattern. EDA is an extended version of basic eventing that integrates systems, subsystems and modules.
EDA introduces vertical integration of systems using events, and horizontal integration of modules and subsystems in the same system using events.
Figure 2 - Horizontal Integration using events
Figure 3 - Vertical integration using events
EDA boosts the loosely coupling of systems, subsystems and modules. EDA does not only talk about integration using events, event processing is also a main topic. EDA contains event generation, event processing, event distribution, execution of the subsequent processes by event clients (sink). EDA categorises event processing into three categories, they are Simple Event Processing (SEP), Event Stream Processing (ESP) and Complex Event Processing (CEP).
Simple event processing distributes events to matching subscribers without any extended processing of those events. This way the distribution mechanism becomes really light-weight and simple and uses matching subscription and event filters. Event stream processing processes real-time data and creates sub sets of events to then publish for the subscribers. Trading data is a good example for ESP. Complex event processing processes events using existing data (events or any other information in master databases) and generates new events or filter incoming events. Generating market data by referring real-time trading data and stored tick data is a good example for CEP.
Figure 4 - EDA Layers
EDA with SOA
Service Oriented Architecture (SOA) brings the capability of building loosely coupled systems using services. In EDA we did talk about integration of loosely coupled systems using events so these two architecture patterns fit each other. In a SOA + EDA integrated system services can invoke when a specific event received by using the data inside the event or a service can create a new event based on the result of an invocation of a service. In the first scenario based on the subscribed topic the target operation of the service can determine and the message payload will be the parameters for the operation. That way filtering of events to pick the service operation will eliminated.
Figure 5 - Service invoke
In the second scenario, Services will invoke by other sources and based on the state changes of the business operation service will act as an event generator and publish events to a event broker system. External event sink(s) that has subscribed to the relevant event topics via a subscription manager will capture the events and do the relevant back end operations based on the content of the events.
Figure 6 - Services as Event generator
To make the EDA and SOA marriage more concrete, the WS-Eventing specification has provided relevant guidelines. WS-E specification describes using SOAP to handle and build a complete publish-subscriber model.
EDA can be used for building sophisticated trading applications. Considering the transaction volumes and real time data involved with a trading system, EDA approch is advantagous in many ways. Assume that a trading system needs to invoke a trading operation in a trading gateway, and there is an order management application that keeps orders and invoke the service based on market values. There is also a sub system that listens to market data and publish market data events based on the best buy or sell. Market data will be published by topics categorized under symbols (e.g. MDD.MSFT, MDD.GOOG). Based on orders received, OMS (Order Management System) will only subscribe to active symbols (topics). When market data arrives for a particulate symbol, OMS will trade those by invoking the operation in the order gateway service.
Document Management System
Assume that an upload service refers to a document repository service to upload documents with document management clients used on user desktops. When a new document gets uploaded, the upload service will call the document service to store the document within the repository. When the document store operation fires (successful call) it publish an event to the topics DOC.INSERT DOC.UPDATE. Client applications subscribed to the particulate event(s) gets notified, and based on client side configurations, relevant operations will be carried out.
Hardware Monitoring System
Eventing is a good fit for a hardware monitoring system. A hardware failure, hardware malfunctioning or the addition of new hardware units to the system can fire events. A subscribed party such as a notification system, a GUI based management console or a disaster recovery system can act based on such events. .
Eventing and EDA are powerfull patterns to achieve loosely coupled systems and subsystems. The combination of EDA with SOA takes that capability to the services spaces and enhances the power of the SOA infrastructure.
EDA gives the capability to easily add and remove services and provides the ability to activate/deactivate operations using subscribe/unsubscribe facilities.
WSO2 Carbon 1.5 brings eventing and EDA capability to the WSO2 SOA stack. In the next article of this series, I will describe technical, functional and implementation details of eventing in WSO2 ESB.
Fusion: Eventing with SOA Part 2 - Eventing using Synapse and WSO2 ESB
 WS-Eventing specification - https://www.w3.org/Submission/WS-Eventing/
 WSDM WEF - https://www.ibm.com/developerworks/library/ac-wsdmgla/index.html
 EDA Introduction - https://en.wikipedia.org/wiki/Event_Driven_Architecture
Asanka Abeysinghe is an Architect at WSO2. asankaa at wso2 dot com