Adding Persistence to Synapse - The Queuing Model

Table of Contents



1. Introduction

2. Who should Read this Article?

3. The Synapse Persistence Model (Queuing Model)

    3.1. Why JMS?

    3.2. Why ActiveMQ?

4. How to Configure Synapse JMS Persistence Model

    4.1. JNDI Configuration

    4.2. Axis2 Configuration

    4.3. Synapse Configuration

5. Running Sample Application

    5.1. The Use Case

    5.2. How to Run Sample

6. Conclusion



1. Introduction

Apache Synapse [1] is a simple, light-weight and high-performance Enterprise Service Bus (ESB). Synapse outperforms most the leading ESB's and comes with a rich set of features that caters for most business requirements [1]. However, Apache Synapse does not posses a mechanism to support message persistence within itself. When it comes to reliable messaging, being able to persist incoming messages is crucial - especially in one-way communications.



Why the one-way case is so special? In a two-way communication channel (i.e. request/response pattern), as the sender awaits for response, he is able to redo any requests in case the messaging hub goes down. Additionally, in a request/response pattern, it is really unlikely that you send a burst of requests and await for responses. So for the most part, it is a single request/response invocation taking place at any given time and thus having a persistence mechanism in the middle may not help so much in the two-way scenario. But when it comes to one-way communication channels, and specially if it is a fire-and-forget kind of message exchange pattern, having a persistence mechanism in the middle would add lots of value to a messaging engine. Typically, a message sender may not re-send a message, unless there is end-to-end message reliability support at the application-layer. As such, should Synapse display support for message persistence, it would greatly enhance its capability for reliable messaging.



There are different flavors of persistence models such as queuing model, flat file model, relational database model, object-relational database model, etc. Queuing model is preferred over other alternatives as it preserves the order of messages automatically. In Synapse, we can get the job done using it's JMS transport, a JMS provider and a combination of two proxy services. We can use ActiveMQ as our JMS provider. This article explains how to achieve persistence in Synapse using the queuing model in detail. Here we go ...



2. Who should Read this Article



It is needless to say that this is intended for Synapse users who are interested in having persistence in Synapse. Without any prior knowledge of the functionality of Synapse you can easily incorporate persistence just by blindly following the set of simple steps given in this article. However, I suggest you have a basic knowledge of JMS [2],configuring Synapse's JMS transport [3] and the concept of proxy services in Synapse [4] so that you'd be able to grab whatever explained here with ease. But either way, feel free to read through and use this as a reference manual to add persistence to Apache Synapse using a queuing model.



3. The Synapse Persistence Model (Queuing Model)

As stated earlier, this persistence model is formed by Synapse JMS transport, a JMS provider and a combination of two proxy services. Figure 1 is self-explanatory. Let's see how it works:

There will be a proxy (Proxy1) accepting incoming messages. Proxy1 makes use of the transport bridging functionality of Synapse [5] so that we can make Proxy1 accept messages that are coming over any transports supported by Synapse. Proxy1 has an endpoint that performs message queuing (JMS), to allow a message received by Proxy1 to be dumped in a queue. We can use ActiveMQ broker to provide us with a JMS queue. This queue is important for two reasons:



    (1) It helps FIFO message flow so that message order is preserved.

    (2) Persistence can be built into the queue to handle message persistence in the middle.

   

    Then, Proxy2 that reads messages from the queue comes into action and pops messages from the queue and pumps them through its in-sequence. We can configure Proxy2 to have desired message mediation functionality.



3.1. Why JMS?



The JMS API deals with a simple queuing model where senders dump messages in queues and receivers read off these queues. Synapse comes with a JMS transport and hence we have a really cool queuing model at hand. We can get Synapse to perform persistence, without adding a single line of code. Believe it or not, adding persistence to Synapse is only a matter of  few configuration changes with care.



3.2. Why ActiveMQ?



Actually, you are not restricted to use ActiveMQ only. Synapse is capable of communicating with any JMS provider and so is our persistence model. But ActiveMQ comes with a really cool feature - support for creating embedded brokers. Why an embedded broker? Let me explain that to you.

In any JMS implementation, we have a central provider that is responsible for managing JMS destinations. So in JMS, we send our messages to predefined destinations and whoever interested reads off the destinations. So in this model, a message has to  travel across two network hops to reach the destination (Figure 2). This is not favorable for two reasons:



    (1) Increased network latency introduced by the two-hop channel.

    (2) Peers are vulnerable to losing connections to the broker. This is a serious phenomenon, as both publisher and subscriber needs to check for the validity of the connection to the broker, and we also have to take measures to make the broker available by adding failover mechanisms.



What if we can have the broker in the receiver itself? In other words, have an embedded broker, so that communication between the server and the broker happens in-memory without going through the network (Figure 3). Then you can have the same queuing model right inside your applications with high-speed message flows. Wow, that sounds great!.

So, if we can embed our JMS broker within the same JVM, then we never have to think of network failures, loosing connections and similar concerns.  We are better off with a JMS provider that can be embedded within the same JVM. Embedding a broker within Synapse alone won't help. As we are using this with our existing JMS transport implementation, the JMS provider should have provisions for creating the broker automatically, just by looking at a parameter given in the JNDI provider URL.



So, ActiveMQ is the preferred choice as it is blessed with both these capabilities. So, until we have a better alternative, ActiveMQ will be our hero!



4. How to Configure Synapse JMS Persistence Model

   

Configuring Synapse to enable persistence is only a matter of configuring Synapse JMS transport and configuring Synapse to have two simple proxies. So let's look at how we can go about doing this in a step-by-step manner.



4.1. JNDI Configuration



First, you need to have a JNDI properties file on your classpath to configure the JMS provider. Our JNDI properties file (jndi.properties) will have the following set of parameters. Actually you can use the attached jndi.properties file as it is.

    

Lets look at what each and every one of these parameters does:



java.naming.factory.initial

    This is the initial context factory to be used by the JVM to create the InitialContext for our JMS. This can vary depending on the JMS provider that you are going to use.



java.naming.provider.url

    This is how you specify where your JMS broker is running. If you look carefully you can see that here the broker URL is not something very familiar to you. Its not http, tcp or anything like that. Here vm:// is used to specify that we are going to use an embedded broker.



connectionFactoryNames

    You can have different types of connection factories and we can give our connection factory a name we prefer. It can be anything and in this sample we have used QueueConnectionFactory.



queue.Proxy2

    This is the logical name that we need our destination (i.e. queue) to be called. This is quite important. In section 4.3 when we look at configuring Synapse, you will notice that we are going to have a proxy with the same name as this destination name. For the time being, let's have a meaningful name for this. Just keep in mind the fact that our Synapse configuration has direct dependency with this destination name. Once we are done with 4.3, you will understand the actual relationship between this setting and the Synapse configuration setting.



4.2. Axis2 Configuration



The only Axis2 configuration that we are concerned about is its JMS transport configuration. Just have a look at the JMS transport configuration section in your axis2.xml. If you are new to using Axis2 JMS, go through [3] to learn how to configure its JMS transport. Here we are going to use default connection factory, so that we will be having the following JMS configuration:



   



Here the connection factory name has to be the factory name we defined in jndi.properties file. And of course it has to be of type queue.



4.3. Synapse Configuration



   

This configures our Proxy1 and Proxy2. A key  factor to note is that both our second proxy and the JMS queue have the same name (i.e. Proxy2).



5. Running Sample Application



5.1. The Use Case



For this, we are going to use the stockquote client that comes with the standard Synapse distribution. So, you need to go through [6] to see how you get the stockquote client to work with Synapse.



We send a number of requests for a quote over HTTP through our Synapse instance to the SimpleStockQuoteService hosted on Simple Axis2 server. Then we stop our Synapse server and run it again. As soon as it comes back up, we should be able to see persisted messages being received by the actual service endpoint.



5.2. How to Run Sample



(1) Place the attached jndi.properties file on your classpath.



(2) Configure the JMS transport listener of your Synapse server as explained in 4.2.

      NOTE : You need to have ActiveMQ installed on your machine. You can get it from [7].



(3) The message flow from client to the actual service endpoint happens in a fraction of a second, that you can not shutdown your Synapse server before all messages pass through it. So to simulate this scenario, we will do the following trick.



        (3.1) Replace your synapse.xml with the attached synapse.xml, change the name of its second proxy to something else (say TempProxy) and start your Synapse server.



        (3.2) Now start your Simple Axis2 server.



        (3.3) Send a couple of requests as follows.

                 ant stockquote -Dtrpurl=https://localhost:8280/soap/Proxy1 -Dmode=placeorder



        (3.4) At this point you cannot see any message being received by the actual endpoint.



        (3.5) Now stop your Synapse server, change the name of the second proxy back to its original name(i.e.Proxy2) and then start the server again.



        (3.6) You should be able to see the messages you sent earlier being received by the actual endpoint



6. Conclusion



Apache Synapse is designed to be a simple, lightweight and a high performance Enterprise Service Bus (ESB) and supports a number of functions out-of-the-box. But Synapse does not have any mechanism to support message persistence within it and hence one might hesitate to use it in his mission-critical application amidst its rich set of features. But we can add persistence to Synapse easily by using its JMS transport, a JMS provider and a combination of two proxy services (say Proxy1 and Proxy2). Proxy1 dumps all the incoming messages in a JMS queue and Proxy2 reads messages off that queue and pump them through its mediator chain. We get a JMS provider that supports persistence so that our JMS queue handles message persistence. Here ActiveMQ is our preferred JMS provider. Not only does ActiveMQ support persistence, it also supports creating embedded brokers so that Synapse can talk to the ActiveMQ broker through an in-memory communication channel.



Author

Danushka Menikkumbura, Technical Lead, WSO2 Inc. danushka at wso2 dot com



References



[1] - Synapse Home Page

[2] - Sun Java JMS Tutorial

[3] - Configuring Synapse JMS Transport

[4] - Synapse Proxy Services Samples

[5] - Synapse Transport Bridging Samples

[6] - Synapse Quick Start

[7] - Download ActiveMQ