2011/11/02
2 Nov, 2011

Implementing Store and Forward Messaging Patterns With WSO2ESB Part 1

  • Charith Wickramarachchi
  • Software Engineer - WSO2

This content last curated on 28/03/2012

Applies to

WSO2 ESB 4.0.3

Introduction

WSO2 Enterprise service bus is a lightweight and high performing enterprise service bus which is recognised for its performance,  user friendliness and 1st class support for enterprise application integration patterns. With the recent WSO2ESB 4.0.0 release it introduced the capability of implementing store and forwarded messaging patterns within  the mediation flow. In the rest of this article I'm going to explain the design of this feature and explain the usages using examples.

Architecture

Implementing Store and forward messaging patters is supported by introducing two new esb constructs named Message Stores and Message processors.

As shown in the diagram Message Store and Message processors are high level configuration configuration constructs.  Message Stores are used to store messages using the store mediator. Message processors can be used to access message stores and process the stored messages.
 

Message Store

As mentioned in the above section Message Store is a top level ESB Configuration construct. Which means All the message stores are defined globally in the ESB configuration. As it name suggests it provides the message storing capability for the ESB. Store mediator can be used to store the messages in the message store.
Message Store is written in a way such that users can implement their own message store implementations and plug them to the ESB. It can be done by implementing; 

org.apache.synapse.message.store.MessageStore

interface or extending
org.apache.synapse.message.store.AbstractMessageStore

Following is the message store configuration

 

As you can see in the "class" section you can specify the fully qualified class name of the message store implementation and plug any message store implementation.
Parameter configuration can be used to pass any configuration parameters that is needed by the underlying message store implementation.

WSO2ESB currently ships two message store implementations.

  1. In memory message store.
  2. JMS message store.

In memory message store

In memory message store used to store messages in the system memory. Due to in memory access it will provide high performance compared to persistence stores. As messages are stored in the memory. Messages will be lost after restarting the server. This message store can be used in the scenarios where we can guarantee that there will be no long term message accumulations (ex : traffic shaping )

 

JMS Message Store

JMS Message store stores messages in a configured JMS queue. So this can be used as a persistence message store.  As messages are stored in FIFO order random access of the messages is not allowed when we use this store. 

 

Message Processors

Message processors are also a Top level ESB configuration construct. Which is used to consume messages in message stores and do the “processing”. The message processing behaviour will depend on the underlying message processor implementation. As an example a Message processor can consume a message in a message store and pass it to an endpoint or sequence. Or it may examine the message content and  take some action etc..

As same as Message stores Message processors are implemented in such a way so that users can write and plug their own Message processor implementations based on the requrement.

Following is the message processor configuration.

 

An important thing to notice here is for message processor configuration we have to provide a associated message store name, which will be used by the message processor as the source of messages.

But that does not necessarily means  a given processor implementation can have only one message store. A Message Processor implementation that need to connect to multiple message stores can use parameter section of the configuration to configure additional message stores.

WSO2ESB ships two message processor implementations named

  1. Message Sampling Processor
  2. Message Forwarding Processor

Message Sampling Processor

Message sampling processor is a Message processor which will consume messages one by one from a configured Message store and inject them to a sequence at a configured rate. Since it sends messages to a sequence, the parameter “sequence” should be defined with the Message Sampling Processor.  

 

Message Forwarding Processor

Message Forwarding processor peeks Messages one by one from the associated Message Store and try to deliver them to a configured Address Endpoint. It will remove the message from the Message store when the delivery is successful. In case of delivery failure it will re-try to deliver the message. Message Forwarding Processor can be configured to get deactivated automatically after a configured number of re delivery attempts. Users can re-try to deliver messages manually from a deactivate message processor by using ESB management console or using JMX.

 

Following is the message forwarding processor configuration.

 

Implementing a Message Throttling scenario with WSO2 ESB.

When it comes to enterprise application integration some times we need to integrate systems that only accepts messages at a given rate or below that. Message throttling can be used in that kind of scenarios to integrate that systems. In this section I'll explain how to implement  a Message throttling scenario using Message Stores and Message Processors.
To implement this scenario I'll use Message Store and Message Sampling Processor.

Following is the flow of the mediation flow.

  1. An ESB Proxy service accepts the message.
  2. It will Store the message in a message store and acknowledge the client with http 202 status code.
  3. A Message Sampling processor will consume messages one by one from the message store and pass it to a Sequence.
  4. That sequence will send the message to an actual back-end service.  

So as you can see the incoming message flow and out going message flow is decoupled using a message store and a processor. As a result Out going message rate only depend on the message processing rate configured in the message sampling processor. (In this scenario we assume that message producer message rate is not always higher than the consumer rate. ex : traffic pattern with massage spikes where average message producer rate over time is below the consumer accepted rate. So that there will be no message accumilation).

Below is the configurations for proxy service, message processor, message store and sequence.

Proxy Configuration.

 

Message Store and Message processor configuration

 
 
Message Sending Sequence Configuration
 
 
In the next article Implementing Store and Forward Messaging Patterns With WSO2ESB Part 2 i’ll explain some advanced configurations and example scenarios that can be achieved with this model.  
 

Author

Charith Wickramarachchi, Software Engineer, WSO2 Inc

 

 

 

 

 

 

 

About Author

  • Charith Wickramarachchi
  • Software Engineer
  • WSO2