[Article] Reliable Messaging With WSO2 Enterprise Service Bus
- Chanika Geeganage
- Associate Technical Lead - WSO2
WSO2 Enterprise Service Bus (ESB) message store acts as a temporary storage in the ESB for messages before they are delivered to their destination by a message processor. This article examines how the guaranteed delivery of messages can be implemented using WSO2 ESB message store and discusses some important sample use cases where this concept can be used.
Table of contents
- Introduction
- What is a message store?
- What is a message processor?
- Reliable messaging through message store and processor
- Usage of reliable messaging
- Use case 1: Reliable delivery of emails
- Use case 2: Processing large messages and reliably deliver to the destination
- Conclusion
- References
Introduction
In the modern enterprise world reliable message delivery is an important aspect in terms of electronic commerce and services. Reliable messaging is the guarantee that a message sent by the sender is indeed received at the other end and received only once on a non-reliable infrastructure such as a network.
What is a message store?
This is used to store messages temporarily before delivering to the destination by the message processor. Your own message store can be implemented using the message store interface and by adding the store to the configuration. WSO2 ESB comes with inbuilt message stores and supports the following stores out of the box:
- In Memory Message Store
- JMS Message Store
- RabbitMQ Message Store
- JDBC Message Store
What is a message processor?
A message processor is used to consume messages that were temporarily stored in a message store and deliver them to the desired backend. This is very useful for sending messages to the backend service when the message rate that comes to the ESB and the rate accepted by the backend service differ.
The inbuilt message processors of WSO2 ESB support the following message processors:
- Scheduled Message Forwarding Processor
- Message Sampling Processor
- Scheduled Failover Message Forwarding Processor
Reliable messaging through message store and processor
The main purpose of using Message Store and Message Processor is to achieve guaranteed delivery or, in other words, reliable messaging. Therefore, when there is a use case that requires a reliable message delivery service over an unreliable infrastructure, in the context of WSO2 ESB, the message store and processor can be used. For an example, there could be scenarios where the availability of the backend service is uncertain, or even though the backend service is available, there could be network failures or application-level failures. Then the message store and the processor can be used to achieve reliable message delivery.
This article will examine some use cases where reliable message delivery can be achieved through WSO2 ESB message store and message processor. A common use case can be found here [1].
Usage of reliable messaging
Use case 1: Reliable delivery of emails
Email is one of the most powerful and effective means of communication in the modern era. But none of your email communication matters if it doesn’t reliably arrive at its destination. For example, a company can rely on email for interaction with its clients, or they might even use emails for e-commerce transactions. If those emails do not reliably reach its destination due to the unreliability of the media, such as network failures, it can even result in loss of business.
WSO2 ESB’s message stores and message processors can be used to ensure reliable delivery of emails even in the event of a failure in the medium, such as network failure, a mail server crash, or system shutdown for maintenance. For this purpose, we can use failover message store and scheduled failover message forwarding processor to ensure the reliable delivery of emails.
Figure 1
As illustrated in Figure 1, we can identify the following message flow between components to achieve this use case.
- The client sends a request to a proxy service, which eventually calls the mail server endpoint.
- But at that time, connectivity between the ESB server and the mail server is lost, so then it sends the message to the failover message store using the message store mediator.
- The scheduled failover message forwarding processor runs periodically and pulls the message from the failover message store and sends it to the original message store.
- When there is a healthy connection between the mail server and the ESB server, the scheduled message forwarding processor sends the email to the mail server endpoint and the mail is delivered to the recipient.
To configure WSO2 ESB server and the message broker, refer to this blog [2].
Use case 2: Processing large messages and reliably deliver to the destination
We can extend the above idea to process large messages, which is a resource-consuming operation, as a background operation and deliver the processed message to its destination without losing the message (apart from delivering the messages to the backend). For example, the message has to be delivered to a web service and after doing some modification to the original messages it has to be written to a database without losing the message. We can integrate the wiretap integration pattern to process a large message with the aid of a sampling processor and two message stores. Further information regarding the wiretap integration pattern can be found here [3]. The reliable message delivery can be achieved through Scheduled Message Forwarding Processor and Scheduled Failover Message Forwarding Processor. Figure 3 illustrates this use case with the components.
Figure 2
As you can see there are two message stores - the first one for the sampling processor and the second one is for the reliable message delivery pattern. Here's what we’ve done:
- The incoming message is copied using the clone mediator and stored in a message store using store mediator.
- Then it takes the message using the sampling processor and will do the required modifications to the messages, such as encoding, adding authentication headers, etc. Thereafter it will pass it to the reliable message delivery.
- The sampling processor calls the failover message store in a case of failure in the second message store due to either network failure, message store crash, or system shutdown for maintenance.
- When the original message store is available, the scheduled failover message processor forwards the modified messages from the failover message store to the second message store.
- Then the messages on the second message store are pulled by the scheduled message forwarding processor.
- Lastly, the scheduled message forwarding processor sends the modified messages to its back-end (to a database for example ).
You can find sample ESB configuration with regard to the wiretap integration pattern in this blog post [4].
Conclusion
In this article, we have discussed how reliable message delivery can be achieved by using WSO2 ESB message store and processor. Further, some useful use cases have been described. This is useful to enterprises as reliable messaging helps to ensure a message sent by the sender is indeed received at the other end and received only once on a non-reliable infrastructure like a network.
References
- [1] https://docs.wso2.com/display/ESB490/Guaranteed+Delivery+with+Failover+Message+Store+and
+Scheduled+Failover+Message+Forwarding+Processor - [2] https://chanikageeganage.blogspot.com/2016/08/guaranteed-delivery-of-emails-using.html
- [3] https://docs.wso2.com/display/IntegrationPatterns/Wire+Tap
- [4] https://shafreenanfar.blogspot.com/2014/03/enterprise-integration-with-message.html
- [5] https://wso2.com/products/enterprise-service-bus/
- [6] https://wso2.com/products/message-broker/