White Paper

WHITE PAPER

12/2019

A Buyer's Guide to API Management SaaS

By Amila Mahaarachchi
Senior Director, WSO2

This buyer’s guide aims to help anyone evaluating an APIM SaaS to make an informed decision. It is a long but worthy read. If you are new to the API management world, we recommend you read this in its entirety. You can also scroll down to the relevant sections if you are only interested in specific requirements. What we believe are compulsory criteria are marked with a [C], while those that are optional are marked with an [O]. However, depending on your requirements, you can decide on the capabilities that are most important to you.

The criteria listed above are included as a table at the end of this guide; the table is also available to download as a fact sheet to use during an evaluation.

Introduction

APIs have been in the technology canvas for nearly two decades now, with almost all the major software vendors releasing their APIs after the year 2000. For example, Salesforce and eBay released their APIs in 2000, Amazon in 2002, and Facebook in 2006. Today, APIs are a fundamental building block in digital business and integrations.

Following their wide adoption, managing APIs has become a critical requirement. This paved the way for off-the-shelf API management products. There are proprietary products, open-source products, and API management SaaS offerings in the API management market, which was estimated at $1.1 billion in 2018.

As noted above, many organizations are looking to adopt APIs, and, during the process, they have to decide on an API management solution to suit their requirements. Selecting such a vendor/solution is not easy owing to a large number of options in the market. This guide should help organizations — particularly those that have made up their minds to offload API management requirements to a SaaS offering — to select a solution by evaluating the capabilities. The guide will not be helpful to formulate a decision on whether a company should opt for an API management product (which will be deployed in their own data center or an IaaS) or an API management SaaS offering. Deciding that is a separate process. Let’s go through each criterion in detail.

Does it support full lifecycle API management?

Full lifecycle API management involves the following key features.

Creating and publishing APIs

Creating and publishing APIs

API owners should be able to create and publish APIs for API consumers to discover and consume them.

Facilitating API consumers

Facilitating API consumers

Once the APIs are published by the owners, API consumers (mainly the application developers who want to integrate the APIs into their application code) need a place to discover the APIs, subscribe, generate tokens and try them out, read the API documentation, etc. This is usually done via a developer portal.

Access control and rate limiting

Access control and rate limiting

Access control is implicit when we discuss API management. Rate limiting is there to protect your backend and also to control different tiers of subscribers who have access to your APIs.

Managing the lifecycle of the APIs

Managing the lifecycle of the APIs

“Managing the lifecycle of APIs” and “Full lifecycle API management” are two things that are misunderstood due to the similarity in wording. The lifecycle of an API starts at its design and ends when the API is retired. In between, there are concepts such as prototyping, versioning. All these are important for a successful API rollout.

API analytics

API analytics

After rolling out an API, it is important to have a mechanism to obtain insights relating to the adoption of the published API. Therefore, API analytics plays a major role in API management. Analytics should provide information on the number of API invocations, the number of subscriptions, geographical locations of origination, latency, faulty invocations, and any other important statistics. This can further be extended to information such as troubleshooting capabilities and logs related to the API invocations.

It is a must for any API management vendor to support the features mentioned above. Most of the popular vendors in the APIM SaaS market support almost all of the above features. If a vendor does not support one or more of the features mentioned above, it will have a negative impact on the vendor in this highly competitive market. Having said that, there may be a limited number of organizations with simple requirements, and they could opt to use an APIM SaaS without a developer portal or without support for capabilities such as prototyping, deprecating, etc. However, even such organizations will be at a loss when their API management programs evolve and have to support more advanced requirements.

What are the security features available?

Security is the most important factor when it comes to API management; it is the main reason behind making APIs managed APIs. When exposing API capabilities to the external world, you need to do it in a secure manner. We discuss security under three main categories.

What are the security features available?
Security by access control

Security by access control

There are two levels at which you can control access to your APIs.

Access control at the API level — This is the most commonly sought security requirement and is achieved via OAuth2 tokens, basic authentication, or by simply using API keys. OAuth2 tokens provide greater flexibility and security as they make it possible to manage the validity of the tokens. There are several grant types that can be used depending on your requirements.

Access control at the resource level — There can be advanced API management use cases that require the capability to restrict access of specific resources to particular users depending on certain criteria. Generally, the scopes concept in OAuth2 is used to achieve this.

Security by rate limiting and filtering

Security by rate limiting and filtering

Even with secured access to APIs, you would still want to protect your backend from high authorized loads. API management vendors provide rate limiting to support this. With rate limiting, you can decide how many requests you want to allow per subscriber and also how many requests should be allowed to reach the backend. You can also decide to drop or deny specific requests by checking the payload values, headers, etc.

Security at the transport layer

Security at the transport layer

There are some advanced use cases where an organization would want to use further security mechanisms, such as mutual TLS. This can be done at two places: communication between the client and the API gateway of the SaaS offering, and communication between the API gateway and the backend service. Some vendors allow you to set up dedicated transport channels, such as VPNs, peered VPCs, direct links, etc., to secure backend services from internet access.

Security covers a vast area and can be a topic of discussion on its own. The details discussed above provide an understanding of the basics to be considered when evaluating the security capabilities provided by a vendor.

Does it satisfy your performance requirements?

Given that the APIM SaaS you are evaluating satisfies the basic requirements mentioned above, it is important to make sure that it satisfies your performance requirements too. The following are some aspects to consider.

  • What is the average response time for your API calls? Is it within the range of your expectations?
  • Can it handle your maximum concurrent load? This is important if your API traffic has a tendency to spike during certain times of the day, week, or month.
  • Does it provide shared runtimes or private runtimes? Some vendors provide shared runtimes and still guarantee proper isolation and performance. Some vendors provide private runtimes per organization. Some vendors have both options available at different pricing tiers and some do not reveal this information.

It would be good to conduct a performance test to make sure the solution will not disappoint you later.

What are the analytics capabilities?

When evaluating an APIM platform, analytics is an important aspect to consider owing to the following reasons:

  • To get an idea on the adoption of your APIs and their usage.
What are the analytics capabilities?
  • To understand trends by analyzing how people consume APIs. This can be done by analyzing the access logs, query parameters, path parameters, etc.
  • To obtain information such as the availability, latency, and faulty invocations of your APIs. This can help you to rectify any issues at your backend service or with your clients’ code.
  • Helps to troubleshoot issues with API invocation.

Vendors invest heavily in providing high-end analytics to users. Some vendors even market and pitch their SaaS offerings based on their analytics capabilities alone. Although analytics is important to any user, you should be careful not to be deceived by it. The importance of analytics can vary depending on your business. For example, if you have only a couple of APIs and your only requirement is to perform access control, then, analytics will not be at the very top of your list of requirements. Conversely, if your business is in retail or is entertainment-related (not limited to these two domains), then you can do wonders by analyzing your API usage.

It is also important to note that analytics is not just presenting graphs and numbers. Alerting and troubleshooting also plays a major part. It may be more accurate to list them under observability, where statistics is also a part of this. When it comes to observability, having the following features can be useful:

  • Configure and receive alerts when things go wrong with your APIs. For example, when your APIs are taking longer than usual to respond, it would be good to receive an alert so that you can look into it.
  • View and download logs to identify any issues with your APIs.
  • The ability to create your own dashboards using the statistics available with the SaaS offering.
  • The ability to pump the logs to your own log analyzing systems to conduct more advanced analysis.
Configure and receive alerts

Pricing of the APIM SaaS

When evaluating an APIM SaaS, the criteria discussed up to now would guarantee technical feasibility. However, just like in everything else, "cost" is a major factor in the decision-making process. Therefore, a proper evaluation of the pricing packages offered by a SaaS vendor is equally important. Owing to high competition in the market, almost all APIM SaaS vendors provide their services at a low starting price. You can launch your API management project by spending just a couple of hundred dollars. But, it is important to understand the different pricing models available. The following are some of the common pricing-related factors, which can be observed with all the vendors.

Pay as you go — All vendors offer some variation of pay-as-you-go pricing. It can be a model that charges based on the number of API calls or a model that allows you to select from a set of tiers, which allow different quotas.

Feature limitations — As a common practice, most vendors restrict the supported features across different tiers. Users have to select a tier depending on the set of features required. This can be disadvantageous when users have to upgrade to a higher tier and pay significantly more to enable a single feature. But, there are a couple of vendors who have deviated from this practice and provide all the available features to customers without any restrictions.

It is clear that the pricing will be pay-as-you-go and you will have to pay for some features you need. The following are a couple of important questions you need to ask yourself when comparing the pricing models:

  • Can you predict your bill with a certain vendor when your API management program grows?
  • Do you have to pay more to handle spikes in your traffic? You may experience spikes once a day or a few times a month. Some vendors handle spikes without additional charges while some will charge more to manage such increases, especially if they have different tiers that support different loads.
  • Is the pricing clear? Are there hidden costs associated, which might mislead you and also hinder your ability to predict your bill? For example, is support included or do you need to pay an additional cost? Are there additional charges such as data transfer charges, etc.
  • What sort of commitment is needed and how easy is the payment process? Most vendors allow monthly payment via credit card. This helps you to start with a low commitment.

SLAs and support

SLAs and support provided by a SaaS vendor are very important when you are relying on them to handle your production API traffic.

Security at the transport layer

A service-level agreement (SLA)

API availability is a key aspect guaranteed by an SLA. Many vendors provide a 99.9% availability of APIs with a guarantee of service credits if they are unable to meet it. This is an acceptable level for 90% of the API traffic scenarios. There are vendors who are ready to provide a more aggressive SLA for a higher price. It is important to check for the following when you compare SLAs.

Is there an incident notification mechanism implemented by the vendor? If not, when APIs are failing due to an incident, engineers would be unaware of the reason for failure. If you receive notifications from the vendor immediately, then you know that the reason could be the incident they are facing and it will help you to handle the situation at your side. A public availability dashboard is another flavor of this. But, sometimes, we have seen that it takes time for vendors to update it during an incident. Yet, it is a good indication of the vendor’s level of transparency.

Support

Support

Support is important due to two reasons.

  1. During the development phase of your project, you will come across situations where you need the vendor’s help to resolve any issues that you come across.
  2. When you are in production, you might face issues that affect your production traffic where you need immediate help to resolve.

Therefore, it is important to know about the support SLA too. For example, how soon do they respond to development type queries?, how soon do they respond to incident type queries?, do they have different support SLAs at different pricing tiers?, what are the support channels available (chat, email, ticketing system)?, etc. Different response times for different pricing tiers will impact your budget too. For example, if it is important for you to get a response for an incident within 10 minutes and if the vendor provides this at a higher tier, it might increase your cost considerably.

Is the SaaS offering available in your region?

Is the SaaS offering available in your region?

If you are in a country with strict data protection laws, you know that you are not allowed to store or route your data out of your country or region. Europe is the best example. There are many other countries with such regulations. For organizations in such countries, it is important that the SaaS offering is hosted in a corresponding region. Most mega vendors who are also in the IaaS business provide this easily while other vendors are catching up too. This is something you need to check based on the laws and regulations you have to adhere to.

In addition, it is also important to have the SaaS offering available in your region because the geographical location of the SaaS offering plays a role in the latency of your API calls. If the traffic originates mainly from your region, it doesn’t make sense for that traffic to go out of the region to meet the API gateways, come back to your data center, go back out of your region again to the API gateways as the response, and then finally come back to the request originated location in your region.

How much white labeling is allowed?

You know you are going to implement your API management program on an APIM SaaS. But, it is obvious that you want it to carry your brand as much as possible. For example, you will want:

  • UIs to reflect the theme of your organization (theming the UIs).
  • URLs to include your domain (custom URLs).
  • Emails going out to the users of the API management program to reflect your brand and email addresses (customizable email templates).

It is always better when more rebranding is possible.

User/Identity management capabilities

Generally, there are three types of users involved in an API management project.

three types of users involved in an API management project

API administrators who manage the project — They create and publish APIs, manage subscribers, view analytics, etc. These are the employees of your organization.

API consumers (application developers) — They sign into the developer portal, subscribe to APIs, and generate access keys to integrate APIs into applications. If the applications belong to your organization, API consumers can be employees of your organization. But, if you are planning to expose your APIs to the public, then API consumers can be external people who will sign up to your developer portal.

End-users of APIs — If your application is generating access keys for each end-user, it means that the APIM SaaS should be aware of them too, i.e., the APIM solution should be able to authenticate the end-users before issuing access keys for them.

Based on the above types of users, the following are some questions to ask yourself when evaluating user/identity management capabilities:

  1. Where are the accounts of my organization’s employees who access the SaaS offering stored?
  2. Can I connect my organization’s IDP to the SaaS offering?
  3. Where can I store external API consumers? Does the SaaS offering facilitate it? If so, is there an additional charge for that?
  4. Where can I store my application’s end users? The same questions in #3 are valid here as well.
  5. Can I plug my end users to the SaaS offering to be authenticated? If so, what security options are available? After all, you cannot just expose your user store to the internet.

Can you monetize your APIs?

This requirement is not valid for every organization. An organization would require an API management program due to following reasons:

Expose APIs for their internal systems’ use — This is seen with organizations stepping into digitizing their business.

Expose APIs for their external partners’ use — This is seen with organizations looking to extend their digitized business to the partners.

Expose data as APIs to the public and charge for the usage — This allows organizations to monetize the data they possess and also encourages innovation with their data.

If you fall into category 3, then this fact will appear at the top of your list. There are vendors who allow you to run an API marketplace in their SaaS.

  • You will publish your APIs and define product plans for the subscribers to purchase.
  • External parties will come and subscribe to your APIs.
  • The SaaS offering will charge subscribers based on the product plan they purchased.

But, it is not as simple as it looks. There are complexities such as:

  • Integrating with different billing providers or payment gateways. Some billing providers do not support every country.
  • Different organizations have different monetization models. This is the toughest nut to crack.

Because of the above reasons, the monetization experience of the APIM SaaS vendors has not been the best yet. Most of the time, you will have to compromise your requirements to match what is available with the vendor or monetize on your own by using the statistics made available by the vendor.

Mediation capabilities

The primary objective of an organization to adopt an API management program is to digitize their business. It involves exposing legacy systems as modern systems, connecting non-compliant systems together, etc. This indicates that it is not just enough for an API management solution to pass through any incoming request to a backend (proxy mode). It is true that for the majority of the cases proxying is enough. However, there is a high chance that you may need to:

  • Change messages coming from clients to match your backend system and vice versa (transforming the message).
  • Have a dynamic routing logic to route requests between multiple backend systems (message routing).
  • Support different content types.
  • Insert parameters, keys, or headers to the message going out to the backend, which can help you achieve certain requirements coming from the backend without changing the client code.

The list above notes just a few types of mediation requirements. There are many flavors of them with varying complexities. It is good to have an understanding of the mediation capabilities provided by the vendor you are evaluating. When organizations find out any limitations related to mediation later in the project, they have to find workarounds to mitigate them or they might end up with a showstopper.

Is a hybrid approach supported?

A hybrid approach to API management is not applicable to all organizations. Even though we are discussing selecting an APIM SaaS, there are valid reasons for an organization to opt for a hybrid approach.

To reduce the latency of API calls

To reduce the latency of API calls

If your backends are hosted in your own data center, it makes sense to keep the API gateway closer to your backend.

Security reasons

Security reasons

You might prefer going for a SaaS for reasons such as reducing the total cost of ownership. But, you might be still concerned about your sensitive data going through a cloud. Also, some data protection regulations might not allow you to send your API traffic out of the country.

To handle internal and external traffic separately

To handle internal and external traffic separately

This means that you use the SaaS gateway for external traffic and a gateway in your data center for internal traffic.

In most cases, a hybrid approach means doing the management tasks in the cloud (the control plane of your API management program) and keeping the runtime on-premises. This gives you significant flexibility and even helps to maintain a multi-cloud API management approach. Note that there are only a few vendors who provide strong hybrid solutions.

CI/CD capabilities

If you are a small organization with a few APIs to be exposed, you may not worry about a release process or a CI/CD process for your APIs. But, if you are a large organization with a lot of APIs and frequent development happening around them, then manual work will be time consuming and error-prone. Therefore, it is good to check what facilities are provided by the SaaS offering to automate your API releases.

Another important aspect that goes hand in hand with CI/CD is having multiple environments. Some vendors do not have a concept of multiple environments. Knowing how you have to handle multiple environments will help you with planning the budget too. The following are a few observations from different vendors:

  • Some do not have an environment concept. You need to create a separate account for a separate environment.
  • Some vendors easily address the environment concept with multi-tenancy.
  • Some vendors facilitate you to move APIs between environments or sometimes even between different deployments of the same vendor, such as between on-premises and cloud.
  • It is good to check how you have to pay for non-production environments, i.e., is it bundled with your subscription or do you have to purchase this separately?

Other important facts to consider

Most of the major criteria you need to consider during your evaluation of an APIM SaaS have been covered so far. It is already a long list, but it can include certain other important aspects as listed below:

Documentation

Documentation

Documentation is a must and almost all vendors provide good documentation, which helps users find the information they need.

Certifications

Certifications

Depending on the nature of your business, your organizational policies, or your government policies, it can be necessary to have a SaaS offering that provides certifications such as ISO, PCI-DSS, and HIPAA.

Frequency of updates

Frequency of updates

It is preferable if the vendor is able to push updates to the system frequently without disrupting the service.

The flexibility of the vendor

The flexibility of the vendor

It is important to understand how easy it is for you to obtain the required improvements or fixes done by a vendor. Note that many SaaS vendors (especially large-scale vendors) are not flexible owing to technical reasons.

Fact sheet

Since this buyer’s guide is a long read, you can refer to the following fact sheet to compare different APIM SaaS vendors. The fact sheet is filled to evaluate WSO2 API Cloud as a potential vendor. It is not filled for any other vendors as a thorough understanding of all of their capabilities is required to do so. To fill this without such knowledge would be unfair to them. Note that some vendors do not maintain the required information as publicly available and you need to contact them to obtain these details. We hope you will find this fact sheet helpful. You can download it below.

DOWNLOAD FACT SHEET
Capability WSO2
Support for full lifecycle API management
Create and publish APIs Yes
Developer portal Yes
Access control and rate limiting Yes
Lifecycle management Yes
API analytics Yes
Security features
Oauth2 token-based authentication for API requests Yes
Other authentication mechanisms for API requests No
Mutual TLS support Yes
Block API requests based on payload Yes
Secure connections from the cloud to the data center Yes. VPN and VPC peering supported
Performance
Does the average response time fall within your expected range?
Can it handle your spikes?
Does it provide shared runtimes or private runtimes? Shared and private both are available
Analytics
API requests statistics Yes
Alerting Yes
View and download logs Yes
Push logs and stats to your own analytics system No. Stats are available via a REST API. Pushing logs is in the roadmap
Pricing
Starting price (per month) $500
Is your monthly bill predictable? Yes
Are there any hidden charges? No
Is support included in the pricing? Yes
Are there feature limitations in different tiers? No
SLA and Support
What is the minimum availability guaranteed? 99.90%
Are there different SLAs for different tiers? No
What is the response time for an incident ticket? Immediately responded.
What is the response time for a query ticket? Within one business day. But generally responded within hours
Availability in your region
Are you able to consume the service in a way your data do not leave your geographical region? Yes. There are API gateways in 7 geographical regions. Can be extended further
White labeling capabilities
Ability to configure custom URLs Yes
Ability to theme the SaaS UI to reflect your brand Yes
Any other white labeling features Theming for email templates
User / Identity management capabilities
Does it allow you to store your external users such as external API subscribers? Yes
Does it allow you to plug your IDP to authenticate SaaS logins? Yes
Can you plug your end-users to the SaaS offering? Yes
Does it allow you to store your API end-users in the SaaS offering? Yes
API monetization
Is there an ability to charge the consumers for your API usage Yes
Can you implement your monetization model with the SaaS offering? No
Mediation capabilities
Is the SaaS offering capable of message mediation? Yes
Is it capable of dynamic routing? Yes
Hybrid API management
Is there a hybrid API management capability? Yes
What components can be run in your infrastructure? API gateway
Which infrastructure is supported by the on-prem components? VMs, Docker, Kubernetes
Support for CI/CD of APIs
Are there APIs to implement a CI/CD pipeline? Yes
Does it support multiple environments? Yes
Are there other options to support CI/CD of APIs? Yes. Export/Import APIs between environments, CLI tool
Certifications
What certifications does the SaaS offering possess? GDPR compliant

Conclusion

With the growing popularity of APIs, it has become important to have an effective API management platform. This buyer’s guide discusses 13 important criteria that you need to consider when evaluating an APIM SaaS offering. Seven of the criteria are marked as mandatory requirements, while six are marked as optional. But, depending on your requirements, this can change. If you are evaluating an APIM SaaS offering, you can download the fact sheet provided above and use it during evaluation. You can also add and remove the criteria to customize the fact sheet depending on your requirements. Although the fact sheet includes a majority of the features you would want to evaluate, there is always a possibility of having additional requirements that you would want to consider.

You can sign up to WSO2 API Cloud, which is WSO2’s APIM SaaS offering, and verify the above capabilities.

For more details about our solutions or to discuss a specific requirement

x

Interested in similar content?