Service Discovery with WSO2 API Microgateway

The world is constantly moving towards microservices architectures. A service that is exposed as an API is actually an aggregation of a set of microservices. This approach helps developers to work in parallel since microservices are loosely coupled. As the exposed service is distributed in different server locations, it improves the system’s availability. However, if consumers are to use microservices as they are, they will have to remember the corresponding IPs of those microservices they need to call; this is inconvenient. From a consumer’s point of view, it would be better to utilize the service as a single component. This is where API management comes into play. From a developer perspective, going ahead with basic functionality — rather than considering other requirements such as authentication, throttling, etc — is preferred. Developers prefer not to expose their endpoints directly. An API manager is responsible for exposing those endpoints to consumers with these additional requirements. Let’s consider an example of an online retail store service. It consists of microservices for adding goods to the store, user management, payment management, etc. The owner of this online retail store requires a  fitting way to approach consumers and develop the business efficiently and effectively.

The Traditional API Management Approach

To accommodate both parties’ requirements, we bring the concept of API management, where developers can implement a large service as a set of microservices while consumers see the same service as a single component. In that case, the entire service is exposed to the consumer through the API manager. Then, the API Manager routes traffic to the corresponding microservices.

Figure 1: Routing Traffic under the traditional API management approach for microservices (static IPs)

The API manager should know about all microservices endpoints. Let’s consider a scenario where a user searches for an item. The user sends a request to a URL, which is resolved to the IP of the API manager.

curl -H “Authorization: Bearer ” https://10.10.10.50:9090/myStore/searchItem

Since the API manager knows about the search-microservice and its endpoint (10.10.10.10:8181), the request is redirected to that corresponding IP. Finally, the response received from the microservice is routed back to the user.

When and Where This Traditional Approach Fails, Within the Context of Service Discovery

Today, most companies in the software industry are moving towards cloud platforms such as AWS, google cloud, and Azure. When your services are running in the cloud, you can scale up or down at any moment by utilizing vertical or horizontal scaling processes. You are only charged for the usage of the purchased resources. So, it is important to purchase and utilize in a cost-effective manner. This means you are dealing with a highly dynamic environment, where services can have different server addresses from time to time. For example, think of a situation where you are running a few services inside a single cluster. However, as traffic increases, you realize that you need to move them to a cluster where more resources are allocated (in terms of CPUs, memory). If you are using a traditional API management approach, you will have to update the API manager separately to route traffic. When there is a large number of services handled by different development teams, it becomes harder to update endpoints manually. Therefore, we need an API management solution where the service discovery is managed by the API manager itself. So far, our concern has been on a production environment. Now, let’s take a look at a developer environment. With the microservices architecture, developers prefer to identify the service at a more granular level. Thus, the service is composed of a set of microservices. And those microservices can be shared between services. In this kind of environment, you have to accept the fact that the server endpoints are subject to frequent change. To accommodate this, you have to consider whether your API management solution can provide service discovery.

WSO2 API Microgateway with Service Discovery

Now we know that change is inevitable for moving your production environment to the cloud. And we could see how crucial is to have the service discovery as a feature. That is why it is better to go for WSO2 API Microgateway along with etcd. Etcd is a consistent and highly available key-value store. To have service discovery enabled, an API developer provides a relevant key as the endpoint in the openAPI, and then, a microservice developer updates the etcd server with the given key and endpoint address. WSO2 API Microgateway can connect to the provided etcd server, figure out the actual endpoint address, and route the traffic accordingly. Let’s go back to the previously mentioned scenario. All the microservices are deployed using separate server instances within the cloud. Imagine a case where, for some reason, the server in which the search-microservice runs needs to be moved to a different server instance. With a traditional API Manager, you will have to update the API Manager with the newly assigned IP address. But, if you are using WSO2 API Microgateway, this burden is removed. You can keep running the microgateway as if nothing has changed.

Figure 2: Routing traffic using WSO2 API Microgateway's Service Discovery

Now, let’s have a look at how this issue is solved in WSO2 API Microgateway. To build the API Microgateway, all we need is the openAPI definition of our online retail store. First, we initialize the microgateway project and add our openAPI definition to the project (For further details, please refer here). Then, we need to edit the openAPI definition. For the “/searchItem” “GET” resource in the Paths object, we need to provide the “x-wso2-production-endpoints” in the following format.

Figure 3: Provide a resource level endpoint using a key

Here, we provide the endpoint URL in the format of “etcd ( <> , <>)”. And the IP address corresponding to the searchItem resource is added to the etcd server as a key-value pair (key = searchItem_etcd_key, value = 10.10.10.10) (Please refer here to gain more insight on how to add/update entries in the etcd server). Then, we can build the deployable artifact by building the project. After all, we can deploy the microgateway with the command line arguments “-e etcdUrl=https://etcdServerIP: port“.

bash gateway -e etcdUrl 10.10.10.60:2379 -e etcdusername=root -e etcdpassword=pass 

The microgateway resolves the actual endpoint IP address for the given resource using the provided etcd key. Whenever it is required to change the server IP address of the searchItem microservice, all you have to do is update the value of the relevant etcd key (in our case, it is search_etcd_key) in the etcd server. The microgateway will automatically pick the value of the changed etcd key so that the client request will be routed to the corresponding server address without any user intervention. This is basically how service discovery is supported in WSO2 API Microgateway. In WSO2 API Microgateway, the user also can distribute the load among several backend endpoints. In other words, Microgateway acts as a load balancer for the given service. Or else, backend endpoints can be configured with a regular endpoint and a failover endpoint. You can use these service discovery features along with those backend configurations. To gain more insight on how to use the service discovery feature, please refer here.