2008/07/25
25 Jul, 2008

Developing Financial Applications with the WSO2 ESB

  • Asanka Abeysinghe
  • CTO - WSO2

Applies To

WSO2 ESB 1.7
Apache Synapse 1.2

Table of Contents

Abbreviations 

OMS Order Management Service
MDD Market Data Distribution Service
Blotter GUI to place orders and order activities
Market Place that the trading happens
EOD End of the Day
SOA Service Oriented Architecture
MOM Message Oriented Middleware
Montage Summarized info of market data for a given symbol for a T amount of time
AMQP Advance Message Queuing Protocol
FIX Financial Information eXchange
Submission Aggregation of an order instruction
POJO Plain Old Java Object
NIO None blocking I/O
CEP Complex Event Processing
Market Report An activity happened in the market for a submission or an instruction

Business Requirement

XYZ is a broker firm that performs equity and options trading. Broker agents place orders on behalf of clients based on the instructions sent by the clients. XYZ broker connects to several market places in the country to post the orders. Upon completion of transaction reconciliations by the end of the day, a broker sends transaction reports to its clients, updating their portfolios and corresponding accounts. Client perform settlements in T+2 (two days grace period given) and the broker payments take place in T+5 (five days after trading date). To make the trading process efficient and real time, XYZ broker has planned to build a system based on the following set of requirements:

The XYZ broker firm needs to build a complete trading platform. Trades will come through an upstream FIX-IN source and brokers will use blotters to post orders manually. Manual orders are generated based on information sent by clients via phone and IM. Brokers split manual orders for submissions and sends them via a downstream FIX-OUT connections to different market places. Orders that come through FIX-IN sources need to be sent to a rule engine in order to find the destination and split count (number of submissions). The rule engine requires montages per stock in order to make trading decisions, and the blotter requires market data to display in the order book. XYZ company connects with a market data service provider to capture real time market data. Trades taking place via the system needs to be persistent, together with executions and the acknowledgments returning from the market. XYZ company needs to send a daily transaction summary to the broker back office system in CSV format, that needs to take place as a part of the EOD process. XYZ is planing to run several instances of the system by dividing orders and market data based on a set of symbols. XYZ is planing to use a SOA architecture with a MOM as the communication channel.

Architecture Without the WSO2 ESB

High Level Architecture

Design Details

Considering the business and technical requirements of the XYZ broker, the above architecture diagram can be used for implementation. This solution architecture use a SOA approach, where functionality is divided amongst services and operations are defined for each of the services. System includes two FIX gateways that service to and from the trading system. A MDD service fetches market data and provide montages, an OMS processes orders and a blotter picks broker agent input in order to generate manual orders. System uses AMQP to have a MOM based communication channel.

FIX-IN Gateway contains an in process FIX engine to maintain FIX-IN sessions and facilitates FIX-IN transports. Once a FIX message reaches the gateway, it converts the message to a POJO and appends it to an AMQP message payload which it then publishes to an AMQP message queue (oms.fix.in,queue). FIX-IN Gateway sends responses (acknowledgments and executions) back to the original FIX source using the relevant sessions.

Similar to the FIX-IN Gateway system, a FIX-OUT Gateway maintains FIX-OUT sessions and sends submissions to various market places that are configured in the system. This gateway takes the responses coming back from market places and hands them over it to the system. Input message comes through AMQP as a POJO and the gateway converts them to FIX messages before dispatch. FIX-OUT gateway consume AMQP messages from a queue (oms.fix.out.queue).

MDD service listens to a market data stream provided by a market data publisher using NIO socket connection. Upon MDD receiving a packet, it publish market data using AMQP topics. A MDD service creates montages considering best bid and best ask - based on buy/sell market data.

Order Blotter provides a GUI for the broker agent to create and manage manual orders. Blotter creates submissions based on order instructions and sends them to the OMS via HTTP-RPC. Blotter consumes AMQP messages to pick market data, order responses, customer portfolio and account information. Blotter has a separate section to view order books that provides room to visually display market data.

OMS processes orders coming through AMQP from the FIX-IN gateway and the HTTP-RPC from the order blotter, processes the orders and creates submissions based on market data and sends them to the FIX-OUT gateway using AMQP. OMS gets market data from the MDD service through AMQP. OMS persists transactions to a RDBMS using a data management layer. OMS creates daily transactions data in CSV format by retrieving data from the RDBMS and uploading text files to the broker back office system using FTP.

Distributed System for Load Balancing

By adding multiplexing capability to the FIX-IN gateway, it routes orders filtered by symbol. FIX-IN gateway publishes relevant orders for different AMQP topics (oms.order.in.<symbol>) and relevant OMS instances subscribe and picks orders for that instance. Symbols are divided among instances, and the number of instances are based on the number of symbols.

Architecture With WSO2 ESB

High Level Architecture

Design Details

In this approach the XYZ broker system has re-architected to use the WSO2 ESB. To get a better idea of configuration information on the WSO2 ESB, please refer the configuration language guide.

Proxy service(s) are configured for FIX transport instead of having two seperate services as was in the previous arcitecture. FIX messages coming through the proxy are sent to OMS via AMQP. The WSO2 ESB does relevant message conversions and converts FIX message into an  XML infoset. XML infoset gets rapped inside an AMQP message payload and is sent to the OMS for order processing. FIX transport layer maintains required FIX sessions and does message correlation using message id and correlation-id that allows the ESB to send relevant executions and acknowledgments back to the original FIX endpoint. FIX transport proxy tunnels orders to the OMS, submission from OMS to the FIX-OUT endpoint, responses from the FIX-OUT endpoint to the OMS and the same responses back to the FIX-IN endpoint from theOMS. FIX proxy puts incoming FIX messages to an AMQP message queue oms.order.in.queue and the OMS consumes FIX messages from that queue. OMS put submissions to an out queue oms.submission.out.queue and the FIX proxy consumes messages from the same. FIX proxy puts responses (acknowledgments/executions) to an OMS in queue oms.mktrpt.in.queue and the OMS consumes responses/market reports from that queue. The OMS put market reports to an out queue oms.mktrpt.out.queue and the FIX proxy consumes market reports and sends them back to the original FIX-IN endpoint.

Order tunneling

Sample configuration (Synapse.config) that configure the above scenario

<definitions xmlns="http://ws.apache.org/ns/synapse">
	<proxy name="FIXProxy" transports="fix">
		<target>
			<endpoint>
				<address
					uri="jms:/QpidStockQuoteService?transport.jms.ConnectionFactoryJNDIName=qpidConnectionfactory&amp;java.naming.factory.initial=org.apache.qpid.jndi.PropertiesFileInitialContextFactory&amp;java.naming.provider.url=repository/conf/sample/resources/fix/con.properties&amp;transport.jms.ReplyDestination=oms.mktrpt.out.queue" />
			</endpoint>
			<inSequence>
				<log level="full" />
			</inSequence>
			<outSequence>
				<property name="transport.fix.ServiceName"
					value="FIXProxy" scope="axis2-client" />
				<log level="full" />
				<send />
			</outSequence>
		</target>
		<parameter name="transport.fix.AcceptorConfigURL">
			file:repository/conf/sample/resources/fix/fix-synapse.cfg
		</parameter>
		<parameter name="transport.fix.AcceptorMessageStore">
			file
		</parameter>
	</proxy>
	<proxy name="oms.mktrpt.out.queue" transports="jms">
		<target>
			<inSequence>
				<send>
					<endpoint>
						<address
							uri="fix://localhost:19876?BeginString=FIX.4.0&amp;SenderCompID=SYNAPSE&amp;TargetCompID=EXEC" />
					</endpoint>
				</send>
			</inSequence>
			<outSequence>
				<send>
					<endpoint>
						<address
							uri="jms:/QpidStockQuoteService?transport.jms.ConnectionFactoryJNDIName=qpidConnectionfactory&amp;java.naming.factory.initial=org.apache.qpid.jndi.PropertiesFileInitialContextFactory&amp;java.naming.provider.url=repository/conf/sample/resources/fix/con-oms.properties" />
					</endpoint>
				</send>
			</outSequence>
		</target>
		<parameter name="transport.fix.InitiatorConfigURL">
			file:repository/conf/sample/resources/fix/synapse-sender.cfg
		</parameter>
		<parameter name="transport.fix.AcceptorMessageStore">
			file
		</parameter>
	</proxy>
</definitions>

Order blotter connects with the ESB using a proxy service that performs HTTP to AMQP transport, and orders coming from the blotter are sent to the OMS using an AMQP message queue (oms.order.man.in.queue). Blotter receives responses back via AMQP. OMS sends customer portfolio information and account information usng AMQP. Blotter listens to the required market data topic mdd.<market>.<symbol>.topic to retrieve market data in order to display them in the order book section. (Sample 251 of the WSO2 ESB samples guide provides detailed information of this configuration)

MDD has disapeared from this architecture by configuring a NIO transport to capture market data from market data source(s), mediate and send it to CEP (Esper has used as the CEP). CEP processes market data based on patterns (EQL) and the ESB publishes events to different market data topics. Market data topics are organized by the market and are filtered to symbol (mdd.<market>.<symbol>). To subscribe only for a certain symbol, the OMS and blotter subscribes for the entire topic mdd.<market>.<symbol> e.g. mdd.nyce.MSFT, to get all symbols from a market mdd.<market>.* e.g. mdd.nyse.*, to get all the symbols from all markets mdd.*.*. ESB puts the symbols in to the AMQP wire by using an AMQP broker (e.g. Apache-QPid). Montages are generated by processing market events using EQL and filtering the best bid and the best ask for a market place. (Blog post 'Event Stream Processing of Synapse' provides detailed information about the Esper integration with WSO2 ESB)

OMS connects with the ESB using AMQP. OMS fetches orders and market reports from two queues and puts submissions and market reports to another AMQP queues that is consumed by the FIX proxies configured within the ESB. OMS does the order processing and creates relevent submissions considering the market events. OMS persists transactions that are used for the reconciliations at the end of the day. OMS uses a state transition module to manage the state of orders and submissions.

EOD process to update the back office is built by configuring a DB mediator to fetch data from the persistent storage and a task to trigger the event at the end of the day. DB records get converted to CSV by using a mediator and sends them back office system using FTP transport. (Sample 360, Sample 255 of the WSO2 ESB samples guide provides detailed information about this configuration)

Trading application built with above architecture is implemented by configuring the WSO2 ESB for various transports and transformations. The entire configuration is done using the in-built Web based console. This user friendly configuration console provides management, configuration and monitoring functionality.

Distributed System for Load Balancing

Two possible approaches can be taken to implement multiple OMS instances running. First, user can configure a simple CBR (Content Based Routing) filter by symbol and route to the relevant OMS instance. Second approch, is to publish orders to CEP as events, and based on the patterns configured, publish filtered events to relevant OMS instances using EPRs. (Sample 1 and 2 of the WSO2 ESB sample guide provides detailed information about CBR).

Summary

With the initial architectural diagram that did not include the WSO2 ESB, a user had to develop all modules related to business services, communication and integration. But with the extended architecture including the WSO2 ESB, all the transportation and transformation functionality related to communication and integration were handover to the WSO2 ESB. This implementation based on the WSO2 ESB configuration language, allows a user to perform rapid changes in terms of improvements and expansions. A user need to build his business services using any language, configure it with the WSO2 ESB and then implement the solution using the extended architecture that uses the WSO2 ESB extensively. The WSO2 ESB's inbuilt support for FIX, AMQP and CEP has made this extended arcitecture a simple and stable solution.

References

[1] Using the WSO2 ESB and FIX

[2] FIX Protocol Org - https://fixprotocol.org

[3] WSO2 ESB Samples - https://wso2.org/project/esb/java/1.7/docs/ESB_Samples.html

[4] WSO2 ESB Configuration Language Guide - https://wso2.org/project/esb/java/1.7/docs/ESB_Configuration_Language.html

Author

Asanka Abeysinghe is an Architect at WSO2. asankaa at wso2 dot com

https://www.asankama.com

 

About Author

  • Asanka Abeysinghe
  • CTO
  • WSO2, Inc