Tag Archives: WSO2 Business Process Server

A Pragmatic Approach to the API Façade Pattern

[Based on a post originally appearing at http://asanka.abeysinghe.org/2013/04/pragmatic-approach-to-api-facade-pattern.html.]

Business APIs expose business functionality for access by external and internal consumers.  In technical terms APIs provide an abstract layer for the internal business services to cater to consumer demand.

Most service platforms are not ready-made to safely and cleanly expose internal services to consumers, posing a common challenge for API providers.  Providing a pragmatic approach to the well-known API Façade pattern is the motivation of writing this post.

Fanike-harajuku-store-opening-1cades hide the complexity of internal implementations and provide simple interfaces.  This is a common pattern in computer science but we can even find it in real world too. Lets look at a real world example first: if you walk to a shoe store you find samples displayed in a manner making them easy to pick and select, but if you walk to the back oUntitled 2f the store you will find a massive warehouse with millions of shoes that will not provide an easy way find the correct shoe for you. What does the showroom do?  It provides a facade that displays the shoes in a way that helps buyers select and buy the shoes they want, thereby reducing the complexity and enhancing the buying experience.

Similarly, facades used in computer systems hide complexity and provide a better experience for the consumers (demand).

FacadeDesignPattern

Lets look at how Façade pattern applies for API Management. There is a clear gap between the consumer demand for APIs and the internal services available in each enterprise. As an example, consumers look for Web APIs that can access using REST principles, deliver content using JSON and secured by OAuth addition to that the APIs can be externally accessible and discover. This may map to an authenticated XML/SOAP based service within the enterprise.

API Façade patterns mainly contains four functional layers:

1. Backend services
2. Mediation
3. Façade
4. External format / Demand Slide08

Most commercial API Management solutions treat the Mediation, Façade and Demand as a single functional, architecture as well as deployment layer.

Slide09

If we look at the gap between backend services and the Demand, can a lightweight mediation layer with limited service bindings such as HTTP/s, JMS can build a business API? Are you willing to add the resulting additional wrapper service layer and maintain it?

WSO2 recommends more pragmatic approach by recommending dividing the Façade layer and Mediation layer into clear functional, architecture and deployment layers.

Slide13

This architecture can facilitate heavy mediation including service chaining/orchestration to provide a business friendly API for the consumers. This also allows one to do a clean deployment by inheriting the infrastructure policies appropriate for each layer, as well as scale each architecture layer separately.

Slide15

Implementation of WSO2 API Façade is facilitated by using the WSO2 API Manager to build the Demand and Façade layers, the WSO2 ESB to build the Mediation layer (also add in products like the WSO2 Business Process Server if required) and connect to the existing services. If you are planning to write new set of backend services using standards such as JAX-WS, JAX-RS you can use the WSO2 Application Server as a runtime. In addition to that if there are any other business/technical requirements to build new business APIs you can add them with or after the mediation layer by leveraging the 17+ products in the WSO2 Middleware Platform.

This pragmatic and architecturally rich approach of WSO2 API Façade pattern results many benefits for the API management solutions:

  • Clean architecture by separating the concerns
  • Have a clear separation of internal and external processing of an API call
  • Ability to scale based on the actual usage of each layer
  • Avoid implementing new services or building wrapper service layers
  • Leverage SOA principles with the new Web API architecture
  • Utilize the middleware and go to market quickly

Having said that if you are planning to have a single layer to facilitate all three layers of API Façade, there are no technical limitations in WSO2 API Management Platform to doing that. You can build the mediation by configuring the pre-configured ESB running as the internal dispatching engine of API Gateway.  However for a real-world deployment we recommend that you consider using the flexible, componentized nature of the WSO2 platform to build a clean, scalable, manageable WSO2 API Façade. In my next post I’ll talk more about how to implement this pattern using WSO2 technologies.

Asanka Abeysinghe
Vice President of Solutions Architecture
Blog: http://asanka.abeysinghe.org

Bus architecture to handle inbound and outbound calls with BPEL

Business processes play a major role in complex, long-running business processes in the modern enterprise. Such business processes might automate such business tasks as ordering and billing, customer or employee account provisioning, financial recordkeeping, auditing, and archiving, supply chain management, and many more.

Within SOA-based solutions a common technology for describing and executing business processes is BPEL (Business Process Execution Language). In an SOA environment, a business interaction with a user or a system results in a call to a Web service representing the business process. Such services may be implemented conveniently using BPEL deployed inside a BPEL engine such as the WSO2 Business Process Server (BPS.)

Since the BPEL engine exposes the process as a service, consumers can invoke the business process using the service interface and whichever of the bindings is most convenient. However, this integration pattern creates a point-to-point connection between the business process and the consumer – something that over time can result in “SOA spaghetti” and make management and evolution of the SOA platform difficult.

The pattern proposed here as a solution avoids this point-to-point connection by introducing a mediation layer using a bus architecture. An Enterprise Service Bus (ESB) presents a face to the consumers, takes the requests to execute the business processes and routes them to the business process services exposed by the BPEL engine. Changes to the system (either the consumers or the BPEL services) can be managed largely within the bus, simplifying versioning, new or alternate protocol deployment, monitoring, security configurations, logging and auditing, migration or scaling of services, etc. The result is more flexibility, greater robustness, and greater insight and manageability of the SOA.

Invoking external services becomes an essential part of BPEL logic. As a result, BPEL engines such as the WSO2 BPS need to connect to various other service endpoints within the service platform.

Commonly, BPEL activities are wired to service endpoints using direct partner links and service endpoint URLs. As a result, point-to-point integration is created between the business process layer and back-end services. These tightly-coupled P2P connections lead to complexity and limit system changes and enhancements, just the problem we were avoiding by fronting the BPEL service with an ESB!

To address this lack of loose coupling for our WSO2 BPS users, we’ve often used a pattern derived from the bus architecture. In a nutshell, this pattern introduces another (conceptually at least) instance of the WSO2 Enterprise Service Bus to mediate between the business processes and the back-end services.

Such a layered architecture looks like this:

Each BPS call that goes to the services layer, does so through the ESB. The ESB invokes the actual services, allowing it to manage all the endpoints and ensures traffic participates in the benefits of routing through the ESB. The diagram above shows two ESB layers. But, in a physical deployment, in most cases, it is deployed as a single ESB instance. Converting the above architecture to a bus architecture helps understand it better. Therefore, lets look at the same thing in a bus architecture.

The above diagram shows that all the upstream consumer channels, business process server (BPEL engine) and the services connect to the same ESB. The ESB wires each component.
This pattern provides a flexible and clean architecture to integrate consumer channels, business processes and backend services.

There are however a few drawbacks to this pattern which need to be balanced with the advantages discussed above. First, this will add a new component to the deployment architecture (the ESB) to be managed in the production environments. Second, two additional layers adding to the communication flows by introducing the ESB may add some latency (typically minimal) to a process invocation. Consider the consequences of these drawbacks when designing your architecture around this pattern.

Asanka Abeysinghe, Director of Solutions Architecture
Asanka’s blog: http://asanka.abeysinghe.org/

Dual channeling for efficient large file processing

Recently I have come to appreciate that a pattern I’ll call “dual channeling” is emerging as a way to address a wide set of scenarios involving large files and workflows with file processing. The Dual Channeling pattern is a variation of the well known enterprise integration pattern “Claim Check”. Recently we helped a customer architect and implement a Dual Channel solution.

Businesses in domains like media/digital media, telco, printing and financial services often require large documents/files to be processed to complete a specific business function. The large file is passed through a series of steps (a workflow). The workflow adjusts to specific document types, clients or jobs. Moving the file in entirety through the workflow steps (which can be many) generally proves to be an inefficient way to manage the workflow. It creates a lot of traffic in the network and increases the time it takes to complete the workflow. Such a process typically looks like this:

sa-blog-6-img-1

The Dual Channel solution avoids this constant shipping of data by introducing two channels, one to carry the actual file and another one to carry the metadata about the file. Many steps in the workflow can then take advantage of a light-weight message with the file metadata to make the decisions and route the workflow. Workflow activities/steps can still call processes that require file processing but in this case, instead of passing the actual file, messages can pass (as part of the metadata) a reference/pointer of the file to the process. A dual channel solution might be represented like this:

sa-blog-6-img-2

To start off the dual-channel pattern, file pre-processing extracts appropriate metadata and ensures clear file identification.

Of course, the Dual Channel pattern can be implemented with the WSO2 Enterprise Service Bus (ESB). The WSO2 ESB acts as a File Transfer Gateway and a Metadata Exchange in this scenario. WSO2 Business Process Server (BPS) can be used to implement the workflows using WS-BPEL. BPEL creation by process designers is simplified with the graphical editor supported by WSO2 Carbon Studio.

sa-blog6-image3

Business process might need to execute rules to fulfill the workflow activities – and in this case the WSO2 Business Rules Server (BRS) is an ideal solution – either as a separate instance or as a feature inside either the WSO2 ESB (where rules are applied to the metadata channel) or WSO2 BPS (where the rules are part of the workflow). Enterprise deployment requirements, high-availability and scalability can be achieved by deploying the WSO2 products in cluster mode using WSO2 Carbon Clustering.

With this pattern, large and complex file processing is more efficient and rapid than ever. As the scope and scale of data explodes in the enterprise, I’m sure more and more enterprise architects will give this pattern a prominent place in their architecture toolbox. I hope it proves useful in yours.

Asanka Abeysinghe, Director of Solutions Architecture
Asanka’s blog: http://asanka.abeysinghe.org/

Defining a Generic API

With a premium placed on loose coupling, a typical SOA deployment displays a high degree of heterogeneity. Different service platforms run in scattered datacenters on a variety of server hardware, operating systems, and development platforms. The services expose different communication and security standards. Individual SOA implementation and maintenance teams will become acclimated to the level of heterogeneity with exposure to the environment, but when an external or internal consumer tries to access the SOA, they will come face to face with this complexity.

image

A common way to simplify and normalize interactions with a heterogeneous environment is to provide a unified API for service consumers — a unified, generic service layer.

One of our commercial bank customers with multiple service platforms began a project of defining a unified services layer, generalizing the the multiple service platforms active in the bank. At first they approached the problem in the traditional way: writing wrapper/proxy services in front of each of the existing services.  As part of an engagement with WSO2 they changed to a “Generic API” solution pattern which dramatically simplified the project by hiding the internal complexity of each service behind a user friendly API, a common URL for service access, and unified security policies.

The “Generic API” pattern installs a common API for the existing service infrastructure, converts traditional applications to services exposed over a normalized set of communication and security protocols, and provides a foundation supporting the easy addition of future service platforms.

image

When implemented with WSO2 products, the Generic API pattern leverages the WSO2 Enterprise Service Bus (ESB) and WSO2 Governance Registry. The WSO2 ESB connects with the back-end service layers and legacy applications, and exposes them through a new service layer.  This is easily accomplished with the proxy service capability of the WSO2 ESB.  Supporting a wide variety of of the transports and message formats, the WSO2 ESB provides a central hub for protocol switching and security mediation between the heterogeneous systems.

With sophisticated transformation capabilities, the WSO2 ESB extends the Generic API pattern to the problem of unifying data models, by converting or mapping messages representing different data models into a common and easily consumed model.

Storing and publishing common metadata such as service descriptions and policies describing the generic API also aids new developers interacting with the system.  In the deployment above, the WSO2 Governance Registry provides a common repository for storing and sharing all the necessary SOA artifacts.

The Generic API pattern provides the foundation for other other solution patterns as well.  In future posts we’ll discuss solution architectures for a Public Services Gateway and an Internal Services Gateway pattern.

Asanka Abeysinghe, Director of Solutions Architecture
Asanka’s blog: http://asanka.abeysinghe.org/

Adding the dynamism of events to a Master Data Management solution

The WSO2 platform provides all the capabilities to address two common architecture patterns — Master Data Management (MDM) and Event Driven Architecture (EDA).

The integration of these two powerful ideas allowed a System Integrator (and WSO2 customer) to refactor and modernize their architecture in their latest release, and roll that out smoothly to their customers.  The new architecture centered around the MDM and EDA patterns.  Built-in facilities enabling MDM and EDA patterns played a factor in choosing the WSO2 SOA-based Middleware Platform.

The existing application software includes a number of RDBMS data repositories, exposed through application-level APIs from various systems. Requirements for the new architecture included the reuse of the existing data as well as support for updates to the existing data stores from messages originating in the new architecture. Even though existing data was reused, the existing data model was not proving a good fit with the new architecture. Therefore converting the data to a new data model also became a key requirement. The MDM pattern fulfilled these two requirements by connecting to the data repositories and converting the data into a universal data model.

The WSO2 platform sports a number of features useful for implementing MDM.  The OxygenTank article Implementing MDM Patterns on WSO2 SOA Platform describes a pattern called Service Adapters that applied neatly to this situation, leveraging the legacy APIs for data access.  The adapters were coded in Java and deployed in the WSO2 Application Server.  WS-Transfer facilitated transformation of the data models and exposed the new universal data model through XML Web Services.

image

The message exchange pattern (MEP) used to integrate the application components was pub-sub (publish and subscribe), bringing EDA into the picture. Pub-Sub extends the loose coupling of a SOA, allowing new data sources to be integrated by a simple publish/subscribe operation.  The WSO2 Enterprise Service Bus’s native support for the WS-Eventing standard allows it to act as an event broker, while extending mediation capabilities to any pub-sub interaction as well as providing all the QoS controls available within the ESB.

By introducing a controller into the architecture, more sophisticated event flows are possible, controlled by business processes and rules. In this architecture, the controller was implemented by using WSO2 Application Server and WSO2 Business Process Server, and combined standard JAX-WS based services and rules defined in BPEL.

Dynamic discovery emerged as a key requirement to avoid tightly coupling of service endpoints.  The combination of WS-Discovery support and a compatible service deployer, endpoint availability is published as each service is deployed.

Integration of a Registry/Repository was identified as a key requirement to store service and configuration metadata as well as to enable dynamic metadata look-up. These facilities are provided by the WSO2 Governance Registry, which in addition to a metadata store hosts the topic store for topic-based event subscriptions.

image

The logical architecture solution above maps to a variety of deployment patterns for different clients of the system integrator, meeting their individual demands for scalability, high availability, infrastructure constraints, and so forth.

The application of aspects of Event Driven Architecture to the problem of Master Data Management adds flexibility and increases the advantages of loose coupling so prized in modern SOA solutions.  We hope the pattern described above gives you some ideas of how your current integration challenges can be approached.

Asanka Abeysinghe, Director of Solutions Architecture
Asanka’s blog: http://asanka.abeysinghe.org/