Orchestration and Choreography - When To Use An ESB vs a Workflow Engine

Service chaining and workflows, commonly referred to as orchestrations, are common integration scenarios in enterprise systems development. There are two distinct type of orchestrations we deal with when realizing these systems:

1. Short-running stateless orchestrations Service chaining messaging patterns that are more synchronous in nature and deals with transient data/sessions.
2. Long-running stateful orchestrations Service chaining messaging patterns that are asynchronous in nature. These workflows take human input and approvals and are configured to run for longer durations with more persistent sessions.

If we look at these two types a bit closely the short-running ones are more straightforward; it mainly follows a request/response pattern, talks to multiple service endpoints, derives a response from one service call and composes the message to be sent to another service call. Enterprise integration patterns, such as message splitting, transformation, cloning and aggregation, are key building blocks in such orchestrations. The enterprise service bus (ESB) pattern fits this description quite well.

However, the long-running ones are more complex in nature and often involve human approval activities, work delegation, and assignments. A typical example for such a use case is a loan approval workflow; when a banking customer applies for a loan, the request triggers multiple services (1) credit approval service - this means connecting to multiple credit bureaus and valuing the customer's credit worthiness, (2) evaluating the loan type (personal/home/vehicle, etc.), (3) finally, human approval (bank manager’s or loan department's approval).

If you look closely, this particular workflow has different types of system interactions from a technical point of view. Not all interactions are long-running, yet the end-to-end scenario is so. The traditional architectural belief for such an end-to-end scenario is to model it with BPEL or BPMN (with a workflow/BPM tool), resulting in a complex model with human tasks and external links.

The common characteristic of this use case is that it is asynchronous in nature. The loan applicant will not get a response instantaneously; other parts, such as calling a credit bureau and getting the credit rating, is a synchronous stateless service call. Manager approval is a human interaction. Hence you do not have to follow the traditional path and build the entire workflow with the BPM tool. What you can follow is a hybrid model where you model the synchronous short-running stateless interactions with an ESB and long-running human interaction with a BPM tool. The proposed solution is depicted in the following architecture diagram.

The corresponding sequence of interactions has been illustrated in the following diagram.