WSO2 API Manager Multi Data Center Deployment Architecture

This has been a hot topic at WSO2 as many of our users and customers often ask how they can deploy WSO2 API Manager in a multi-datacenter setup. This blog post is an effort to elaborate the technical details and internals of the multi data center (m-dc) deployment architecture.

Caching, clustering, deployment synchronization, worker-manager separation, API subscription/metadata sharing and traffic routing are some of the important areas engineers and devops have to focus on at the deployment stage. Each of these aspects will be explained in detail to provide more clarity.

Caching  and clustering

The WSO2 platform uses Hazelcast as the clustering framework/engine, which is also a JSR-107 (JCache) provider. The platform has support for L1 and L2 cache, and the L2 cache is implemented as a distributed cache that adds and removes values from a Hazelcast distributed map. More information about WSO2 caching implementation can be found here:

When it comes to a multi-dc deployment, WSO2’s recomendation is to set up the cluster local to a data center. This is done mainly to avoid environmental stability issues (such as the cache not getting synced instantaneously) due to network latencies across geographic locations.

Worker-manager separation

This is the traditional setup where we will carry out the worker-manager deployment per data center. Each data center will have its own manager node managing the local cluster.

Deployment synchronization

Deployment synchronization in a multi-dc deployment is two-fold: the local synchronization between nodes and across data center artifact synchronization.

In this step, among the data centers, a master needs to be selected. Even though there are multiple manager nodes within each data center, only one manager is configured to check-in artifacts (<AutoCommit>true</AutoCommit>) other managers will not commit anything. The data center with the privileged manager node will be the master data center.

We also have to setup SVN repositories at each data center and only the master will have a read/write repository whereas others will be read-only. There needs to be some mechanism (there are plenty of tools for this task) to synchronize these SVN repositories in a unidirectional manner.

Once new APIs are added from the master data center’s manager, it will take some time to synchronize across data centers because there won’t be a cluster message broadcasted across DCs to get an update; therefore, the nodes in slave data centers will eventually be consistent (artifacts will be polled periodically from the SVN). This can be expedited if required by reducing the polling interval and the SVN sync delay.

API publishing

API publishing will only happen from the master data center, hence publisher application will only be deployed at the master.

API subscription and metadata sharing

When an API is published there are associated metadata like tags, throttling information, scope information, comments, and ratings. An API also has subscriptions coming in from both data centers; this means OAuth tokens, keys, and secrets. All this information is stored in the WSO2 registry database schema and API manager database schema. These schemas need to be replicated across the data centers. Traditionally, tools like Oracle Goldengate are used for this task and if it’s in EC2, the RDS replication can be used.

API traffic routing

Since the data centers will be eventually consistent, enabling session affinity (sticky sessions*) at the load balancer is the best option to avoid intermittent resource errors that are not found. This will also avoid any throttling inconsistencies that can occur in a multi-dc setup.

Complete deployment architecture


* Also Refer

SOA patterns and an enterprise middleware platform – a winning combination!

Service-oriented architecture (SOA) patterns provide structure and clarity, enabling architects to establish their SOA efforts across the enterprise. Moreover, these SOA patterns also help to link SOA and business requirements in an effective and efficient way.

Since service orientation has past roots in distributed computing, some of these patterns share similar attributes with distributed computing patterns and concepts. SOA design patterns also inherit concepts from other areas, such as object orientation patterns, enterprise architecture integration patterns, enterprise integration patterns, and software architecture patterns.

SOA design patterns capture the essence of past best practices, solution design principles, and general guidelines to build efficient SOA systems. Even though some of the implementation approaches of SOA keep changing (e.g. the introduction of rapid application development, the importance of devops principles in SOA and gradual movement from a top down approach to a bottom up approach etc.), these principles and design patterns make a lot of sense when building small to large, simple to complex distributed and service oriented systems.

The WSO2 Middleware Platform provides a set of loosely coupled, lean, open source capabilities that can be mixed and matched to build an end-to-end SOA solution. This platform approach means architects have the power of a range of orthogonal tools that support open standards to build their solution – the open standards based integration between the components is a strong arsenal when building solutions since architects can now bring in any preferred standard compliant component to achieve certain functionality, in line with an organization’s enterprise architecture principles.

The WSO2 team presented a series of SOA patterns, aptly categorised as the ‘SOA Patterns Webinar Series’ in 2014. Each webinar in this series covered the definition and explanation of the said pattern, along with real world use cases, solution architecture principles, and examples on how the pattern could be achieved using the WSO2 middleware platform and its suite of products. This is just the tip of the iceberg though – WSO2’s orthogonal toolset means implementation of patterns are just a download away.

SOA Pattern: File Gateway

Presented by Mifan Careem and Jason Catlin, this pattern looks at how a File Gateway pattern can be used to interact with real time and batch processing systems, by providing an intermediate file processing service between batch files and services. This webinar also looks at related patterns such as service loose coupling, data format transformation, service abstraction and technology and operational challenges in implementing this.

SOA Pattern: Policy Centralisation

Suresh Attanayake and Umesha Gunasinghe look at the importance of managing organizatinal policies centrally to overcome redundancy issues and inconsistencies and how a Policy Centralization pattern can help overcome this. This webinar looks at how the WSO2 Platform, focusing on the WSO2 Identity Server, can be used to implement such a pattern.

SOA Pattern: Legacy Wrappers

Chintana Wilamuna and Nadeesha Gamage explore the well known Legacy Wrapper pattern, as a solution to an enterprise which has large amounts of customised legacy code that cannot be easily modified nor replaced. In this webinar, they also look at common and popular tools from the WSO2 toolset for providing a legacy wrapper, including the WSO2 Enterprise Service Bus and the WSO2 Data Services Server.

SOA Pattern: Asynchronous Queuing

As business ecosystems become more critical, real time and complex, users expect inter-system messages to be processed in less than a second, regardless of the complexity and distance between ecosystems. Senaka Fernando and Lakmal Kodithuwakku look at the Asynchronous Queuing pattern as a solution to this. In this webinar, the team also focuses on the pros and cons of such a pattern as well as the implementation details using the WSO2 platform.

SOA Pattern: Event driven messaging

In a connected business context, systems need to work efficiently with each other, responding to internal and environmental changes real time. An event-driven architecture for messaging provides a solution to this by allowing the external entities to establish communication channels to the main entity as subscribers for its events. Dakshita Ratnayake and Chathura Kulasinghe look at the Event Driven Messaging pattern in detail, along with a demo of the same in this webinar.

SOA Pattern: Compensating Service Transaction

In this webinar, Nuwan Bandara and Nipun Suwandaratna look at different strategies of compensating transactions and how they can be used to recover systems to the original abstract states to guarantee system integrity. They discuss a solutions approach to achieve business integrity across stateful and stateless transactional workflows and how the WSO2 platform can be used effectively to achieve this.

Event-Driven Architecture for Online Shopping

Event-driven architecture (EDA) offers high agility and expandability to integrate with future applications while providing real-time analysis and monitoring as events occur, ensuring that today’s solutions will also meet long-term requirements.

WSO2 offers a full suite of open source components for both event-driven SOA architectures and Web services architectures to implement highly scalable reliable enterprise grade solutions. WSO2 is one of the only vendors that can deliver all components of the EDA and Web services  architectures. WSO2 is also open source and built to be enterprise grade throughout.

Another common use case of EDA is online shopping because it can vary considerably in complexity depending on the scale and ways in which goods can be sold or acquired, and the process of fulfillment. A typical example of an online retailer is presented in the diagram below.


In this architecture, consumers have a possibility to communicate through a mobile app or go to a website to purchase goods. When they use a mobile app it can talk directly to the ESB. When coming in through a Web service it will typically initiate a process in an app server.

All information goes through the ESB, so requests to search, look for more information, place orders, query the status of orders, etc. are all processed through the ESB and lead to initiation of business processes or directly querying the database and returning a result.

A business process will coordinate fulfillment, determine if there is inventory or where the inventory is, and kick off a back-order process if required, which may then trigger processes to inform the customer about delivery dates. Shipping may be notified in a warehouse to initiate a delivery.

In this architecture, we assume the suppliers have an API to interact with the selling merchant so they can inform the merchant of their delivery and to place orders. Real-time inventory must be managed in the RDB and product information constantly ingested and updated.

Activity monitoring is used to collect data on all activities throughout the system, including the customer, so that metrics and big data can be analyzed. A CEP processor is included so real-time offers can be made to customers if analytics determine it would be beneficial. RDB is used with MB to log transactions and other mission critical data.

Event-Driven Architecture and the Healthcare Industry

Insurance companies, state health care systems, and HMOs need to manage the health of customers and provide medical decisions. There are 4 parts of such a system, which is often referred to as an MMIS system. The key components of an MMIS system are as follows:

  1. Provider – enrollment, management, credentials, services enrollment
  2. Consumer – enrollment, service application, healthcare management
  3. Transactions, billing, and service approvals
  4. Patient health data, big data, health analysis, and analytics

Each of these systems are integrated and each requires its own event-driven architecture (EDA). Standards in the health industry include HL-7 for the message format and coding. Important standards to be supported in any system include HL-7, EHR standards, ICD coding standards, and numerous other changing specifications. Systems need to support strong privacy, authentication, and security to protect individuals privacy.

Let’s look at one particular type of healthcare transaction; the enrollment process for patients in an insurance service. A typical enrollment system for consumers would include at least the components depicted in Figure 1.

A typical enrollment system for consumers-01-01

Figure 1

When a patient requests to enroll in a medical insurance company or system they typically make an application in one form or another. To facilitate this numerous ways are provided for the applicant to submit the information.  As a best practice, this application uses an ESB to mediate and transform whatever application source is used. Mobile applications, for instance, can talk directly to the ESB. Once an application has been received it needs to be reliably stored and a business process initiated to process the application. Typically, the patient’s past data will have to be obtained from existing medical systems as well as history of transactions, payments, providers, etc. so that a profile can be created to determine if the application should be approved.

Over time, new information coming into the system may undermine an applicant’s eligibility to participate in a certain plan. Hence, the system has to continue to ingest data from various data sources including information on the applicants living address, medical conditions, and behaviors. A CEP engine can detect events that may trigger a business process to review an applicant’s status.

WSO2 offers a full suite of open source components for both EDA and Web services architectures to implement highly scalable and reliable enterprise grade solutions. It is typical to use both architectures in today’s enterprises. WSO2 is one of the only vendors that can deliver all components of both architectures. WSO2 is also open source and built to be enterprise grade throughout.

Figures 2 illustrates a view of WSO2’s connected health reference architecture.

WSO2’s connected health reference architecture-01

Figure 2

In the architecture described by the above diagram, health care organizations have integrated event driven capabilities. A Data Services Server helps collect raw information that can be processed by analytics tools for learning and detection of anomalous conditions. Healthcare privacy is a key requirement and the architecture above provides the security required.

The Use of Event-Driven Architecture in Trading Floors

One of the first use cases for publish/subscribe event driven computing was on a trading floor. If you consider the typical architecture of a trading floor, it comprises information sources from a variety of providers. These providers aggregate content from many sources and feed that information as a stream of subject-oriented feeds. For instance, a trader who focuses on the oil sector will subscribe to any information that’s relevant that will likely in the traders estimation have an impact on prices of oil securities. Each trader would have a different view on what affects oil securities or the type of trading they do; therefore, even though you may have 2,000 traders on your trading floor, it’s unlikely that two of them will be interested in the same set of information or how these are presented.

The Use of Event-Driven Architecture in Trading Floors-figure-01-01-01-01

Figure 1 (Source: Cisco)

Building a trading floor using EDA architecture involves building a high-performance infrastructure consisting of a number of services that must be able to sustain data rates well in excess of 1,000 transactions/second. As explained in Figures 1 and 2, ultra high reliability and transactional semantics are needed throughout. Every process is provided in a cluster or set of clusters and usually an active/active method of fault tolerance is employed. Message broker (MB) is used for trades and things related to auditable entities. Topics are used to distribute market data. Systems are monitored using an activity monitor and metrics produced. Data also needs to be reliably sent to risk analysis, which computes credit limits and other limits the firm has for trading operations in real-time. Complex event processing is used to detect anomalous events, security events, or even opportunities.

The Use of Event-Driven Architecture in Trading Floors-Figure-02

Figure 2

WSO2 offers a full suite of open source components for both event-driven EDA and Web Services architectures to implement highly scalable and reliable enterprise grade solutions. It is typical to use both architectures in today’s enterprises. WSO2 is one of the only vendors that can deliver all components of both architectures.

WSO2 is also open source and built to be enterprise grade throughout.

In a high-frequency-trading application (HFT), specialized MBs are used to minimize latency to communicate to the stock exchanges directly. A bank of computers will pull market information directly from sources and high powered computers will calculate opportunities to trade. Such trading happens in an automated way because the timing has to be at the millisecond level to take advantage of opportunities.

Other applications are for macro analysis that usually involves complex ingestion of data from sources that aren’t readily available. A lot of effort is put into data cleansing and a columnar time-series database that understands the state of things as they were known (prior to being modified by data improvements). These are called as-of data and involve having persistent all variations of data and modifications so the time-series can be recreated as was known at a certain time. Apple uses such notions in its Time Machine technology where calculations involve running historical data through algorithms to determine if the calculations will produce a profit or are reliable.

Online Taxi Service – A Typical Use Case of EDA

Event-driven architectures (EDAs) are sometimes called messaging systems. A message is simply an event or vice versa, an event becomes a message. The concept of an event-driven system is that everything that could benefit is notified of these events simultaneously and as soon as possible. Thus, the earliest real-time event driven systems came up with the notion of publish/subscribe.

An online taxi service is a typical use case of EDA; it has several applications that all talk directly to an ESB hub in the cloud or an API management service that deliver messages in real time between interested parties.

A sample online taxi service potential architecture is depicted in the figure below. case-study-Ufer-Taxis

As illustrated here, a message broker is added for queueing and creating a publish subscribe framework in the backend infrastructure. This allows a new pickup to alert several support services and tracking. There’s also an API store for external developers who want to integrate the Ufer service into their apps, making it easier to arrange pickups or drops from any location.

WSO2 offers a full suite of open source components for both event-driven SOA architectures and web services architectures to implement highly scalable reliable enterprise grade solutions. WSO2 is one of the only vendors that can deliver all components of the EDA and Web services  architectures. WSO2 is also open source and built to be enterprise grade throughout.