WSO2 API Manager: Embracing Event-Driven Architecture

Please note that the content is applicable for API Manager version 3.x series.

WSO2 API Manager, the leading open-source hybrid API management platform, released the latest version recently; the new iteration comes with a number of powerful new features and product improvements. This article does not examine these new additions, but instead focuses on the architectural enhancements.

We will start by looking at what was there in the pre 3.2 era and then move on to observations on usage, which led to the new architecture. We will then dig deeper into the newer architecture and wrap up by looking at a few deployment pointers, specifically for existing deployments.

WSO2 API Manager was event driven in the first place

WSO2’s API Manager is modular in its architecture with different components addressing the main key capabilities in the product. The following diagram provides an overview.

Figure 1: WSO2 API Manager—Key Components

The component that I would like to focus on is Traffic Manager, which is used to process API throttling policies and provide these details to the gateway. Initially, prior to Traffic Manager, the gateway processed throttling policies. This gave rise to a few drawbacks such as:

• Affecting gateway latency as processing occurred at the gateway on invocation
• Not being able to perform throttling based on any other parameter other than the number of invocations such as bandwidth
• Not being able to handle complex throttling policies; only serve simple rate limiting

In order to address these issues, Traffic Manager came into the picture. I’ve written in detail about (the then new) Traffic Manager in my blog. To summarize, this is a node that has a real-time event processing engine and a message broker—both necessary components that were needed to fill the missing functions. With the real-time processing engine, it became possible to efficiently process traffic-related events in real-time and also to do much more complex event processing, thus enabling support for complex throttling policies. With events being processed in real-time, there arose a need to convey event results in an efficient manner. This is where a message broker comes in. Events needed to be sent asynchronously without having any impact on gateway latency and the results were needed to be subscribed to many gateway nodes in a deployment. So, the introduction of the Traffic Manager component to the product’s architecture was the first step in making some parts of the underlying architecture event-driven to provide better capabilities.

Let’s now fast forward to WSO2 API Manager 3.2

Traffic Manager was introduced with version 2.0 in June, 2016. Since then, WSO2’s API Manager has been deployed at many customer sites as the main interaction interface for digital transformation in an enterprise. In some cases, the deployment remains small, i.e., limited to two nodes in HA serving traffic. However, in most cases, we see the product deployed across multiple data centres, on various deployment modes (plain HW, VMs, and containers), on-premises, and on the cloud. Over the years, we have observed the following.

• When an invocation happens at the gateway, a series of validations occur. These can be broadly categorised into three main areas: subscription validation, scope validation, and token validation.

A subscription to an API happens at the developer portal, and, once this is done, the details are stored in the database. Similarly, when a token is requested from the token endpoint via the gateway, a record of this token is stored in the database. If the token is an opaque token, the entire token is stored; if the token is a JWT token, then the jti value is stored. In order to perform these validations, the usual way was for the gateway to connect to the key manager and then the key manager would look at the database and decide whether the subscription and token were valid. This caused the gateway to wait for this validation to happen and also rely on another component to do this. A cache was introduced to mitigate this, and the call to the Key Manager was made if there was no cached decision or when the cache invalidated. However, the dependency of the gateway on the key manager still remained. By introducing self contained JWT tokens, the need for validating all tokens were removed, and, with this capability, which came in with API-M 3.0, gateways became more self-sufficient. However, there was still a DB lookup needed to check on the subscription validation and scope validation and this needed to be improved.

• With a dependency on the key manager (when a deployment architecture started to scale based on usage as the number of gateways increased), there arose the need to increase the number of key managers. This made scaling expensive.
• WSO2 API Manager deploys artifacts to gateways once an API moves from the created to published state. The deployment artifact was a file system artifact, which required this to be managed properly across all gateways. If multiple versions of this artifact exist at the gateways, then the API will not behave the same at all gateways (which is the expected norm). So, in order to handle this, we used techniques such as manager-worker patterns, where the manager makes sure the artifacts are distributed to all nodes having a shared file system across the gateways to which the publisher pushes the artifacts into. However, this resulted in issues such as adding more nodes to a deployment and making it complicated, not being able to have a shared file system in certain situations (e.g., in containerized environments), etc. It was time to move away from file-based deployment synchronization of artifacts.
• WSO2 API Manager’s Key Manager (KM) component is the inbuilt OAuth server, which is used for providing API security. Enterprises sometimes have their own OAuth server where all the API users are already registered at. In situations like this, WSO2 API Manager provided an extension point through which this external OAuth server could be integrated into the inner workings of the product, so that all requests starting with OAuth application creation when the subscription process begins (such as token issuance and introspection) are performed at this external OAuth server. However, in some instances we observed that, when there are multiple departments using the same global API platform, each department can have its own OAuth server. Moreover, some enterprises needed to provide API authentication and authorization through multiple OAuth servers depending on which API was being used. This made a clear statement that multiple key manager support was a basic requirement (and no longer a nice add-on feature) and that gateways needed to know about these configurations per API.

Taking the above factors into consideration, new capabilities were added into WSO2 API Manager 3.2. These include:

• Providing in-memory subscription validation
• Providing an inbuilt artifact synchronization tool
• Providing the ability to define multiple external key managers for tenants and APIs separately.

When bringing in these features to the product, we evaluated the existing architecture and saw that an event driven mechanism was needed to incorporate these changes (e.g., while some components relied on publishing data, others needed to listen in to these events and then adjust accordingly). So, the existing eventing architecture was expanded and Traffic Manager was made the central eventing hub, since it already had the required components.

The expanded eventing architecture of WSO2 API Manager 3.2

Let’s take a look at the various activities that became event publishers with these changes and what components came into the picture.

The publisher and the developer portal become the main two components that publish events in this expanded eventing architecture, with traffic manager becoming the intermediary broker and the gateway becoming the recipient of almost all of these events.

• API creation, update, and deletion - This is a task done at the publisher, and on each of these actions, events are fired to the traffic manager to indicate changes. The publisher still saves this information in the WSO2 API Manager database. This also includes details such as API lifecycle management state changes and scopes and policy additions or updates against APIs and API resources. This will also indicate the key manager configurations for each API.
• Application creation, update, and deletion - This is a task done at the developer portal and events pertaining to these actions are again fired to the traffic manager as well as details being stored in the database.
• Subscription creation, update and deletion - This is done at the developer portal when subscribing to APIs and this information is also first fired to the traffic manager and then saved to the database. Figure 2 depicts all of these points.

Figure 2: Publisher and Developer Portal updating Traffic Manager on API/App CRUD actions

• When third-party key managers are configured and added, these details are stored in the database and also sent as events to the traffic manager by the admin portal as shown in the figure below. The event data is a mapping of the identifiable token variable to the respective key manager validator.

Figure 3: All Portals updating the Traffic Manager on Key Manager actions

• If inbuilt artifact synchronization is configured, then all artifacts that eventually are deployed in the file system at the gateways will first be stored as blobs in the selected storage (e.g., database) by the publisher. The corresponding events are fired to the traffic manager together with the gateway labels they apply to. Steps numbered 1 and 2 in Figure 4 depicts this point.

Figure 4: Traffic Manager’s role in inbuilt artifact synchronization

The embedded message broker within the traffic manager receives these events and routes these to the various topics defined.

The multiple gateways deployed subscribe to these topics and whenever a new event comes in, they will receive a notification on that subscription channel. At the gateway, a series of maps, as shown in Figure 5, are maintained in memory to store all this information, so that at runtime, the details in these maps can be used for subscription validation, key manager determination, etc.

Figure 5: Gateway details being used at invocation

If for some reason a gateway is not up and running at the time an event is published, it will access a REST API at the traffic manager to update itself with all relevant information on startup. This action also happens if the gateway finds a particular map being empty; it will contact the traffic manager via the REST API and update itself. This avoids any errors that may have occurred due to network failures between the traffic manager and the gateway. This mechanism enables new gateways in the deployment to become aware of the current configurations in an efficient and secure manner. This behaviour of the gateway is shown in figure 6 below.

Figure 6: New or offline gateways getting updated when active

If inbuild artifact synchronization is configured, then, as soon as a notification event is received, the gateways will filter out those events that are relevant to them through the labels associated with the events and load the artifacts from the common storage into the gateway’s file system. Steps numbered 3 and 4 in Figure 7 depicts this point.

Figure 7: Gateway getting updated when inbuilt artifact synchronization is active

With these changes, the interactions between different components of WSO2 API Manager can be summarized into a single diagram as shown in Figure 8.

Figure 8: Overall interactions between all components

Deployment implications of event-driven architecture in WSO2 API Manager

With these changes in the product, there are a few implications that need to be noted.

• Traffic Manager becomes a first-class component of the product. Previously, it was a component that was only mandatory if throttling functionality was used. However, by being promoted to being the central messaging hub in the new architecture, the traffic manager becomes a mandatory component.
• Traffic manager needs to have a connection to the AM databases now. Previously, it was only connected to the database used for the message broker; however, now it connects to the main databases. This implies that the traffic manager node needs to be placed within a militarized or secure zone in the network when deploying a solution.
• With the gateway using a request path to the traffic manager to update in memory tables the HTTP port (9443) should be made available so that the REST API can be invoked when needed.
• There is no impact in the gateway-to-traffic manager ratio, which was used to validate deployments before these changes. Most of the above-mentioned traffic happens at design time and not run time. So, the gateway-to-traffic manager ratio at run time still remains as 10:1.
• The impact of the gateway scaling up on the Key Manager goes down with the new architecture as almost all of the validations are done at the gateway itself, such as subscription validation, scope validation, and self-contained token validation. The token introspection of the Key Manager will only be needed for legacy-opaque token invocations. With this, even though the gateway scales up, the Key Manager does not need to scale up unless there is an edge case, where the deployment uses only opaque tokens and where the number of invocations are very high. Do note that the Key Manager is still needed in the deployment since it provides the capability of the resident KM, providing underlying security needs for the portals, etc. It is just that it does not need to scale as much as it used to.

Try it out today!

To explore all these powerful features, download the latest version of WSO2 API Manager and follow the official product documentation to get started. Moreover, you can find more resources on YouTube and Medium. If you want help from the community, head over to the Slack channel.