In this article I will discuss the architectural design and implementation details of tenant aware API management. While it did not support multi-tenancy in previous versions, WSO2 API Manager 1.4.0 onward will have multi-tenancy and be compatible with other WSO2 multi-tenant products.
|WSO2 API Manager
||Version 1.4.0 and above
Table of Contents
- Why multi-tenancy for API management
- Data level isolation for tenants
- Execution level tenant isolation
- Implementation of multi-tenancy
- Distributed deployment
According to resources  multi-tenancy refers to a principle in software architecture where a single instance of the software runs on a server, serving multiple client organizations (tenants). Multi-tenancy is contrasted with a multi-instance architecture where separate software instances (or hardware systems) are setup for different client organizations. With a multi-tenant architecture, a software application is designed to virtually partition its data and configuration, and each client organization works with a customized virtual application instance. Multi-tenancy is regarded as an essential attribute of cloud computing.
In summary, multi-tenancy is to have one instance but customizable service for each tenant isolated from one another. Tenants may be given the ability to customize some parts of the application, such as color of the user interface (UI) or business rules, but they cannot customize the application's code. Multi-tenancy is economical because software development and maintenance costs are shared between all tenants. Also, small organizations that do not want to maintain their own infrastructure or data centers, can simply use a deployed multi-tenant cloud service. This way they are able to get isolated environments for their organization.
02. Why multi-tenancy for API management
The WSO2 API Manager was a single-tenant application in releases up to version 1.3.1. So it can be used only for a single organization with data shared among all users. Even though it is a single-tenant application, we can create user groups and set permissions and visibility as needed. We can limit operations and visibility of API applications according to the permission of the given user. But eventually every user belongs to the same domain and all share the same data. To overcome this limitation we came up with a multi-tenant model for the WSO2 API Manager.
Here are a few use cases where we may need to deploy API Manager in a multi-tenant manner -
Let say an organization has a few departments such as engineering, accounting, finance, etc. And all these departments may have a set of APIs for department specific tasks. Employees in that particular department may develop and use their own APIs. Also in some cases they may need to share their API with other departments as well. For example the HR department can expose employee details API to all other departments. So all department can access employee data within the entire organization. For this example we can map a tenant in to a department.
Another use case is where an organization can deploy API manager as a service and let consumers to create their own API store. This way smaller companies do not need to maintain their own infrastructure they can simply create an account and have their own API store. Multi-tenancy will help implement this type of use cases. In this example each small company maps to a tenant. The small company named foo can register for a tenant domain foo.com. Then they can have their specific API store and publisher
Let say you have a tenant named test.org where all users that belong to test.org can see and use APIs only created by test.org tenant users. So if you logged in with [email protected] you will see only APIs created by your domain users. But in some cases you may want to make your APIs visibile to the public as well (users of wso2.org tenants should able to see your APIs). In that case we have the public API store concept. This is why we have two modes - Public API store and Tenants API store. Here we are more focused on user level experience and the critical part is making the store and gateway tenant aware. Other than this, its almost the same as tenant-aware registry and ESB. Now any user can see his tenants APIs and all public APIs.
To address above use cases we need multi-tenancy. Data level isolation and execution level isolation was the main challenges we found while implementing multi-tenancy. Lets see how we achieve data and execution level isolation for different tenants.
03. Data level isolation for tenants
Usually all API metadata is stored in registry space. So we decided to store tenant APIs in tenant registry space. WSO2 Governance Registry supports multi-tenancy and we used the same for this as well. So all APIs created by tenant will be stored into this registry space.
When a user logs into the API store we should show him all APIs he can consume.
API store and publisher runs as super tenant deployed software-as-a-service (super tenant deployed SaaS). When API store and publisher runs they operate as super tenants apps. Those apps can do anything super admin can do. According to the design of WSO2 registry we can retrieve tenant registry if we have super tenant permission. So these store and publisher apps consume each and every tenants registry as super admin of system. When new tenant logs into the system (store or publisher applications) we (store and publisher app code) will load that tenant's registry as super admin. From that point onwards every API related operation is running against that tenant registry space. If tenant user creates a new API it will be stored in that particular tenants registry space. A tenant cannot access another tenant registry and create anything or consume it. This is how we achieved data level isolation between tenants.
Data level isolation is very important because an organization might not want to share everything with other organizations in the same space but in some cases may need to publish some APIs and public APIs (everyone can see and consume it). Also we allow users to see public APIs even if they didn't log into the system. In a global API store we need to retrieve all public APIs available. For this we need to access other tenants APIs. So here we will iterate through each tenants registry space as super admin and get public APIs. We call even getAll API method as tenant underlying code block will execute as super admin and return with all public APIs.
Figure 01. How data level isolation works in multi tenanted API manager
04. Execution level tenant isolation
The next most important thing is execution level isolation. As we described earlier, the store and publisher apps are super tenant deployed SaaS. Each time the user logs into the system we will create object API, API Manager which is specific to that user. In that case we will get tenant domain for that user and initialize it. For each API Manager object we need a registry object. So we will use tenant registry for that. Then that tenant cannot access any other tenants data. Inside the API Manager implementation we need to do some validations and authorizations against the user. For this we are using tenants authorization manager.
Rating and comments are other most important runtime data available in API management. In previous versions of the WSO2 API manager we stored rating and comment data along with API data. But later we realized that comments and ratings are not part of the API and that they belong to runtime stats. So we moved them to a new table. From WSO2 API Manager 1.4.0 onwards comments and rating will be stored in a separate table inside the API Manager database. Other advantages of this change is that the tenant does not need to access/write other tenants registry data. If we stored rating/comments with API data belonging to Tenant A , then other tenant users - Tenant B, who rate/comment on it have to write in Tenant A registry space.
Application and subscription related data stored in API manager database. API manager code allow to access those tables and modify them. Tenants cannot consume those data directly. Tenant users can create applications and request for new token. Underlying API management code will handle those request and respond accordingly.
05. Implementation of multi-tenancy
WSO2 API Manager statistics are sensitive runtime data sets we collect. If we enable stats for API manager, API invocation data will persist into WSO2 BAM. Before we introduced multi-tenancy into WSO2 API Manager we pushed stats without the tenant domain. From version 1.4.1 onwards, stats are publish to WSO2 BAM2 with tenant domain. So when we do a summary generation, the tenant domain also would be part of it. If some user clicks view stats button on the publisher UI then statistics request goes with tenant domain. So users will see only relevant stat data (data specific for logged in tenant).
Multi-tenant API gateway is almost the same as multi-tenant ESB. In this case each tenant will get an isolated environment for their artifacts. Its available under /repository/tenants. Inside this folder we will have a set of folders specific to tenants with tenant ID. When a new tenant is loaded into system configuration, context listener (available in API Manager implementation) will copy necessary sequences to the tenant’s space. This includes API Manager specific sequences for throttle, sandbox check, resource mismatch, auth validation, and build sequence.
WSO2 API Manager comes with everything you need to deploy in production environment, including gateway, key manager, store, and publisher. In production deployments, its highly recommended to deploy them in a distributed manner. Here we need to do a server to server communication. All server to server calls use super tenant credentials. Even users logged in as tenants, all server to server calls are made by super tenant as this is a super tenants application.
Its important to have tenant aware urls for each and every resource we are accessing. Other tenant aware WSO2 products follow /t/tenant.org/ pattern for urls. But API manager apps are working as super tenant deployed SaaS. In that case we cannot have /t/tenant.org pattern. So we will use query parameter to specify tenant domain. Let say we have tenant aware load balancer and we need to forward requests to correct tenant cluster. In that case we should embeded some tenant sensitive data in the url. So the load balancer can identify the tenant domain and forward to the correct cluster. When we do redirections in publisher store apps, tenant domain will append to url as query string eg:/store/?tenant=tenant.org.
Multi tenant WSO2 API Manager allows small business users to maintain private API stores without having their own infrastructure. Service providers can deploy API manager service and let users to have private API store and publisher in an isolated environment. This will be productive and cost effective solution for all API related service providers.
Sanjeewa Malalgoda, Software Engineer, WSO2 Inc.