2010/09/25
25 Sep, 2010

Transactional ESB (ESB transaction use cases)

  • Rajika Kumarasiri
  • Senior Software Engineer - WSO2

Applies To

WSO2 Enterprise Service Bus(ESB) 3.0.1

Introduction

Transaction plays a major role in today's business world. It adds a very important functionality to your system, which you will only see in a system crash. Having good transactional support in your system will definitely save your business in a system crash. The first section is an overview of transaction support in WSO2 ESB. Then a walk through of three complete samples, where the transaction mediator and the JMS transaction support in WSO2 ESB will be introduced. These samples will explain how you can use WSO2 ESB transaction support in a real world scenario in your production system.

Table of Content

What is a transaction?

A transaction, is a set of operations executed as a single unit. It also can be defined as an agreement which is carried out between separate entities or objects. A transaction can be considered as indivisible or atomic when it has the characteristic of either being completed in its entirety or not at all. During the event of a failure for a transaction update, atomic transaction type guarantees transaction integrity such that any partial updates are rolled back automatically.

Transactions have many different forms, namely Financial transactions, Database transactions, etc. In the ESB point of view we can think of two types of transactions.

  1. Distributed transaction.
  2. JMS transaction.

Distributed transaction

A distributed transaction is a transaction that updates data on two or more networked computer systems. It extends the benefits of transactions to applications that must update distributed data. Implementing robust distributed applications is difficult because these applications are subject to multiple failures, including failure of the client, the server, and the network connection between the client and server. For distributed transactions, each computer has a local transaction manager. When a transaction works at multiple computers, the transaction managers interact with other transaction managers via either a superior or subordinate relationship. These relationships are relevant only for a particular transaction.

JMS transactions

JMS local transaction

A local transaction represents a unit of work on a single connection to a data source managed by a resource manager. In JMS, we can use the JMS API to get a transacted session and call methods for a commit or rollback for the relevant transaction objects. This is managed internal to a resource manager. There is no external transaction manager involved in the coordination of such transactions.

Distributed JMS transaction

An external transaction manager manages the coordination of the transaction. Designing and using JMS distributed transactions is more complex than using local JMS transactions.

The transaction manager is the primary component of the distributed transaction support infrastructure, however the JDBC driver (the resource adapter) and the application server (in which you deploy your applications) should have the following two characteristics.

  1. The driver should implement the JDBC 2.0 API(including the optional package interfaces XADataSource and XAConnection) or higher and the JTA interface XAResource.
  2. The application server should provide a Datasource class that is implemented to interact with the distributed transaction infrastructure and a connection pooling model for performance improvements.

When JMS transactions are in place local transactions are managed by the JMS provider itself whereas the distributed JMS transactions are managed by the XAResource enabled transaction manager in the J2EE application server. Also note that you need to check if your application server provides a XAConnectionFactory when you look for the ConnectionFactory. So if you plan to run the following samples with different database and J2EE server environment, check for the above requirements (also this is a MUST that you need to verify if you plan to run a transactional system).

If you need an in-depth knowledge in transactions (distributed transactions, JMS local and distributed transaction etc.) refer [1].

WSO2 ESB transaction support

WSO2 ESB has two kinds of transaction support.

  1. Transaction mediator
  2. JMS transport transaction.

Transaction mediator

A new Synapse mediator (transaction mediator) has been added which supports distributed transaction using Java transaction API (JTA)[2]. JTA allows applications to perform a distributed transaction that is, transactions that access and update data on two or more networked computer resources (an example would be to have two databases or a database and a message queue such as JMS). This mediator can be used to perform a distributed transaction. The Synapse configuration has been extended to add explicit transaction markers. What this mean is you can use the Synapse configuration language to define the start, end etc., of your transaction. Its the responsibility of the user to define when to start, commit or rollback the transaction. For example we can mark the start of a transaction at the start of a database commit and end the transaction at the end of the database commit and we can mark rollback transaction in case of a failure occurring.

Transaction mediator configuration

<transaction action="new | use-existing-or-new | fault-if-no-tx | commit | rollback | resume"/>

The transaction action attribute has the following meanings.

  1. New - Create a new JTA transaction. Generate a fault if a transaction already exists.
  2. Use-Existing-or-New - Create a new JTA transaction. Do nothing if a transaction exists.
  3. Fault-if-no-tx - Generate a fault if no transaction exists. Do nothing if a transaction exists.
  4. Commit - Commit transaction. Generate fault if no transaction already exists.
  5. Rollback - Rolls back the pending operations for the session. Generate fault if no transaction already exists.
  6. Resume - Resume transaction. Generate a fault if no transaction exists.
  7. End - Indicates the end of a transaction for the session.

Each of these actions can be used to mark the required action of the distributed transaction.

Transaction mediator scenario

We can use the following scenario to show how transaction mediator works. Assume we have a record in one database and we want to delete that record from the first database and add it to the second database (these two databases can be run on the same server or they can be in two remote servers). The database tables are defined such that the same entry can't be added twice. So in the successful case the record will be deleted from the first table (of the first database) and will be added to the second table (of the second database) and in the failure case (in which case the record which is going to add is already in the second database) no record will be deleted from first table and no record will be added into the second database.

System Requirements

  1. Windows, Linux or Solaris operating systems.
  2. WSO2 ESB 3.0.1 - [3]
  3. JBoss Application server (jboss-5.1.0.GA version used) - [4]
  4. Derby Database (db-derby-10.5.1.1-bin version was used) - [5]
  5. Apache ActiveMQ 5.3.1

Since the transaction mediator is implemented using JTA you need to have a JTA provider. We'll be using JBoss J2EE application server (which implements the transaction support through Arjuna TS) as the JTA provider, so we'll have to deploy WSO2 ESB in JBoss Application server(AS). Apache Derby as the database server. The References section contain the links to download the required softwares. Please note that JBoss server and the Derby database server support the characteristics mentioned above.

Running the example

  1. Unzip WSO2 ESB distribution to a place of your choice. And then remove the geronimo-jta_1.1_spec-1.1.0.wso2v1.jar ( This jar file can be found in $ESB_HOME/repository/components/plugins). The reason is we are using the implementation of javax.transaction.UserTransaction [6] of JTA provider (here JBoss) and if we have both in class path there is a classloading issue which causes the transaction mediator not to work.
  2. We need to deploy WSO2 ESB on JBoss AS. See [7] to see how this can be done. The JBOSS installation path will be referred to as $JBOSS_HOME and the WSO2 ESB repo location as $CARBON_HOME.
  3. Drop the derby client jars (derby.jar, derbynet.jar and derbyclient.jar) into $CARBON_HOME/repository/components/lib folder and also into $JBOSS_HOME/server/default/lib (we'll be using the default JBoss configuration) folder.
  4. We'll be using a similar sample like #361[8], and the full Synapse configuration is show below (you can directly paste the following configuration into synapse configuration in $ESB_HOME/repository/conf/synapse-config/synapse.xml). In the in sequence we'll send a message to the service and in the out sequence we'll delete an entry from the 1st database and will update the 2nd database with that entry. If we try to add an entry which is already there in the second database the whole transaction will be roll-backed.
    <definitions xmlns="http://ws.apache.org/ns/synapse">
       <sequence name="myFaultHandler">
            <log level="custom">
                <property name="text" value="** Rollback Transaction**"/>
            </log>
            <transaction action="rollback"/>
            <send/>
        </sequence>
        <sequence name="main" onError="myFaultHandler">
            <in>
                <send>
                    <endpoint>
                        <address uri="https://localhost:9000/services/SimpleStockQuoteService"/>
                    </endpoint>
                </send>
            </in>
             <out>
                <transaction action="new"/>
                <log level="custom">
                    <property name="text" value="** Reporting to the Database esbdb**"/>
                </log>
                <dbreport useTransaction="true" xmlns="http://ws.apache.org/ns/synapse">
                    <connection>
                        <pool>
                            <dsName>java:jdbc/XADerbyDS</dsName>
                            <icClass>org.jnp.interfaces.NamingContextFactory</icClass>
                            <url>localhost:1099</url>
                            <user>esb</user>
                            <password>esb</password>
                        </pool>
                    </connection>
                    <statement>
                         <sql>delete from company where name =?</sql>
                         <parameter expression="//m0:return/m1:symbol/child::text()"
                           xmlns:m0="https://services.samples" xmlns:m1="https://services.samples/xsd"
                                     type="VARCHAR"/>
                    </statement>
                </dbreport>
                <log level="custom">
                    <property name="text" value="** Reporting to the Database esbdb1**"/>
                </log>
                <dbreport useTransaction="true" xmlns="http://ws.apache.org/ns/synapse">
                    <connection>
                        <pool>
                            <dsName>java:jdbc/XADerbyDS1</dsName>
                            <icClass>org.jnp.interfaces.NamingContextFactory</icClass>
                            <url>localhost:1099</url>
                            <user>esb</user>
                            <password>esb</password>
                        </pool>
                    </connection>
                    <statement>
                        <sql>INSERT into company values (?,'c4',?)</sql>
                        <parameter expression="//m0:return/m1:symbol/child::text()"
             xmlns:m1="https://services.samples/xsd" xmlns:m0="https://services.samples"
                                   type="VARCHAR"/>
                        <parameter expression="//m0:return/m1:last/child::text()"
             xmlns:m1="https://services.samples/xsd" xmlns:m0="https://services.samples"
                                   type="DOUBLE"/>
                    </statement>
                </dbreport>
                <transaction action="commit"/>
                <send/>
            </out>
        </sequence>
    </definitions>
    
  5. You'll need to have two distributed Derby databases (esbdb and esbdb1) to run the sample. Refer here [8] on how to set up the databases. The database table was created using the following sql query. Note the table schema in which we can't have the same entry twice.
    CREATE table company(name varchar(10) primary key, id varchar(10), price double);

    Add few records to the two database table

    Database1:

    INSERT into company values ('IBM','c1',0.0);
    INSERT into company values ('SUN','c2',0.0);

    Database1:

    INSERT into company values ('SUN','c2',0.0);
    INSERT into company values ('MSFT','c3',0.0);

    Note the order of the record

  6. You also need two data source declarations for JBoss AS for the two distributed database.

    Datasource1:esb-derby-xa-ds.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <datasources>
        <xa-datasource>
            <jndi-name>jdbc/XADerbyDS</jndi-name>
            <isSameRM-override-value>false</isSameRM-override-value>
            <xa-datasource-class>org.apache.derby.jdbc.ClientXADataSource</xa-datasource-class>
            <xa-datasource-property name="portNumber">1527</xa-datasource-property>
            <xa-datasource-property name="DatabaseName">esbdb</xa-datasource-property>
            <xa-datasource-property name="User">esb</xa-datasource-property>
            <xa-datasource-property name="Password">esb</xa-datasource-property>
            <metadata>
                <type-mapping>Derby</type-mapping>
            </metadata>
        </xa-datasource>
    </datasources>
                    

    Datasource2:esb-derby1-xa-ds.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <datasources>
        <xa-datasource>
            <jndi-name>jdbc/XADerbyDS1</jndi-name>
            <isSameRM-override-value>false</isSameRM-override-value>
            <xa-datasource-class>org.apache.derby.jdbc.ClientXADataSource</xa-datasource-class>
            <xa-datasource-property name="portNumber">1527</xa-datasource-property>
            <xa-datasource-property name="DatabaseName">esbdb1</xa-datasource-property>
            <xa-datasource-property name="User">esb</xa-datasource-property>
            <xa-datasource-property name="Password">esb</xa-datasource-property>
            <metadata>
                <type-mapping>Derby</type-mapping>
            </metadata>
        </xa-datasource>
    </datasources>
                    

    Just drop the above two datasource declarations into (note that the two datasource file name should be *-xa-ds.xml ) $JBOSS_HOME/server/default/deploy folder. You also need to map the above jndi names, so drop the following jboss-web.xml configuration into : $JBOSS_HOME/serer/default/deploy/esb.war/WEB-INF/

    <!DOCTYPE jboss-web PUBLIC
             "-//JBoss//DTD Web Application 5.0//EN"
             "https://www.jboss.org/j2ee/dtd/jboss-web_5_0.dtd">
     <jboss-web>
         <resource-ref>
             <res-ref-name>jdbc/XADerbyDS</res-ref-name>
             <jndi-name>java:/XADerbyDS</jndi-name>
         </resource-ref>
         <resource-ref>
             <res-ref-name>jdbc/XADerbyDS1</res-ref-name>
             <jndi-name>java:/XADerbyDS1</jndi-name>
         </resource-ref>
     </jboss-web>
                    
  7. Now go into $JBOSS_HOME/bin and start the server. Just run the run.sh (run.bat) script. Note that you need to set the CARBON_HOME enviornment varible pointing to the Carbon repository location.
  8. Now we can try the samples. See sample set up guide[9] to see how you can set up the server. We need to deploy the SimpleStockQuote service which comes with WSO2 ESB sampels.

    Successful scenario:

    First we'll remove the IBM record from the 1st database and add it to the 2nd database. Run the sample with following options.

    ant stockquote -Daddurl=https://localhost:9000/services/SimpleStockQuoteService 
    -Dtrpurl=https://localhost:8280/ -Dsymbol=IBM

    Check both databases to see how the record is deleted from 1st database and added into the second database.

    Failure senario:

    Now we'll try adding an entry which is already there in the second database. This time use Symbol SUN.

    ant stockquote -Daddurl=https://localhost:9000/services/SimpleStockQuoteService
    -Dtrpurl=https://localhost:8280/ -Dsymbol=SUN

    You'll see how the fault sequence is executed and the whole transaction is rollback. Check both database again. You'll notice that no record is deleted from the first database and no record is added into the second database.

JMS transport transaction

In addition to the transaction mediator, WSO2 ESB also has support for JMS transaction. WSO2 ESB ships with a JMS transport (which support JMS 1.1 and later) which supports both local and distributed JMS transactions. In a JMS client, you can use local transactions to group message sends and receives. The JMS API Session interface provides commit and rollback methods that you can use in a JMS client.

JMS local transaction

WSO2 ESB fully support JMS local transactions.

Scenario

In the following scenario a message will be read from a JMS queue and it'll be processed by a back end service. While executing one sequence ESB will receive a fault and this will cause the JMS transaction to rollback. In the successful scenario the transaction will be committed and the request will be sent to the back end service. In the following configuration there is a class mediator which will set a property called "MESSAGE_COUNT" to the message context and it will demonstrate this behavior. Depeneding on the value of this varible the decesion to either commit or rollback the transaction will be taken. You can download the binary zip of the mediator from here. Drop this mediator to $ESB_HOME/repository/components/lib.

The source of the mediator looks as shown below.

            public class MessageCounterMediator extends AbstractMediator {
                private static int MESSAGE_COUNT = 0;

                public boolean mediate(MessageContext synCtx) {
                    MESSAGE_COUNT++;
                    synCtx.setProperty("MESSAGE_COUNT", MESSAGE_COUNT);
                    return true;
                }
            }
        

ESB configuration is given below.

<proxy name="StockQuoteProxy" transports="jms" startOnLoad="true">
        <target>
            <inSequence>
                <class name="org.wso2.carbon.mediator.MessageCounterMediator"/>
                <switch source="get-property('MESSAGE_COUNT')">
                    <case regex="1">
                        <property name="SET_ROLLBACK_ONLY" value="true" scope="axis2"/>
                        <log level="custom">
                            <property name="Transaction Action" value="Rollbacked"/>
                        </log>
                    </case>
                    <default>
                        <log level="custom">
                            <property name="Transaction Action" value="Committed"/>
                        </log>
                        <send>
                            <endpoint name="endpoint_urn_uuid_677F3EF4BC0AE1AF5B32399295906279-2025938318">
                                <address uri="https://localhost:9000/services/SimpleStockQuoteService"/>
                            </endpoint>
                        </send>
                    </default>
                </switch>
                <property name="OUT_ONLY" value="true"/>
            </inSequence>
        </target>
        <publishWSDL uri="file:repository/samples/resources/proxy/sample_proxy_1.wsdl"/>
        <parameter name="transport.jms.ContentType">
            <rules>
                <jmsProperty>contentType</jmsProperty>
                <default>application/xml</default>
            </rules>
        </parameter>
</proxy>
            
        

To start a local JMS transaction you define the following property in JMS transport Listner in axis2.xml

<parameter name="transport.jms.SessionTransacted">true</parameter>

By default the session is not transacted and if you want to use JMS local transaction, set the above parameter to true. Also note the following property in the failure case which will roll back the local transaction.

<property name="SET_ROLLBACK_ONLY" value="true" scope="axis2"/>

Running the example

Note that you don't need to deploy the ESB on JBOSS to run this sample since this does not require any distrbuted transaction manager as in other cases.

  1. Copy the JMS client jars (activemq-core-5.2.0.jar, geronimo-j2ee-management_1.0_spec-1.0.jar, geronimo-jms_1.1_spec-1.1.1.jar) into $ESB_HOME/repository/components/lib
  2. Start the activemq server.
  3. Deploy and start the SimpleStockQuoteService service. You need to get the SimpleStockQuoteService service attach with this article and (SimpleStockQuoteService.aar) and place it in $ESB_HOME/samples/axis2Server/repository/services.
  4. Start WSO2 ESB with above configuration
  5. Run the JMS client with following command.
    ant jmsclient -Djms_type=pox -Djms_dest=dynamicQueues/StockQuoteProxy -Djms_payload=MSFT

When you run the client you can see two log lines saying that first time the traction was rolled back and in the second attempt the transaction was committed. And service will be served to the message received.

Distributed JMS transaction

WSO2 ESB has the support for distributed JMS transaction. In this case you can use the transaction mediator to manage multiple distributed resources. An ideal candidate for this catagoery would be handling a JMS queue and a data base server using a single transaction. A sample configuration will be very similar to the distributed transaction example configuration given in the section on distributed transactions.

Conclusion

A transaction, is a set of operations executed as a single unit. It is also known as an agreement, which is carried out between separate entities or objects. It is indivisible or atomic, when it has the characteristic of either being completed in its entirety or not at all. WSO2 ESB provides support for transactional ESB which includes Transaction Mediator and JMS Transport Transaction. Transaction Mediator is based on Synapse, which supports distributed transaction support using the Java transaction API. JMS Transport Transaction is based on JMS 1.1 and later, which support both local and distributed JMS transactions.

Future work

Some efforts are in progress to add the support for using a stand alone JTA provider such as Atomikos[10].

References:

  1. https://www.amazon.com/Java-Transaction-Processing-Design-Implementation/dp/013035290X
  2. https://java.sun.com/javaee/technologies/jta/index.jsp
  3. https://wso2.org/projects/esb/java
  4. https://www.jboss.org/jbossas/downloads/
  5. https://db.apache.org/derby/derby_downloads.html
  6. https://java.sun.com/products/jta/javadocs-1.0.1/javax/transaction/package-summary.html
  7. https://wso2.org/library/knowledge-base/2010/03/deploying-wso2esb300-jboss510ga
  8. https://wso2.org/project/esb/java/3.0.0/docs/samples_setup_guide.html#derby
  9. https://wso2.org/project/esb/java/3.0.0/docs/samples_setup_guide.html
  10. https://www.atomikos.com/

Author

Rajika Kumarasiri, Senior Sofware Engineer,WSO2 Inc

 

About Author

  • Rajika Kumarasiri
  • Senior Software Engineer
  • WSO2 Inc.