How to Achieve API Chaining (Batch API) with WSO2 API Manager

Applies to

Table of contents

1. Introduction
2. Required Services
3. Setting up environment
4. Understand and test scenario
5. Conclusion

Introduction

Let’s consider credit service as an example. When we send a request to the credit service, we need to provide all information required to obtain credit (client details, credit amount, etc.). However, sometimes we do not have all of these details about the client when we make this request. For instance, if we only have the client’s name or ID when he/she made a credit request, we would need to talk to person information service and retrieve additional required information about this particular user. We might have service or API, which returns user information when we provide a user ID. We can call personal information service/API with the client ID and obtain all required information, like address, full name, age, etc. Thereafter, we can make a request to credit service with all information required. When this requirement comes to the API world we need to follow the same pattern. In addition, we might have credit service that would require all user information and credit amount. Furthermore, we would need to expose a simple credit service to our client, which would only require a client ID and credit amount. Now let’s see how we can achieve this with WSO2 API Manager.

Required services

In this example, we have two back­end services (deployed on WSO2 Application server) and three APIs deployed in WSO2 API Manager. WSDLs and source codes for the examples can be found in the attached zip file.

Credit API

This service is available in the WSO2 API Manager and is exposed to customers. This service accepts an ID and a credit amount for its credit operation. A request coming to this service is served by two back­-end services.

PersonInfoService

The PersonInfoService provides the name and address of a requestor when the ID is given. So, this is the first service being called by the CreditProxy service deployed in the API Manager.

Personal Information API

API created by pointing to PersonInfoService hosted in WSO2 Application Server. An external user will not be able to directly invoke actual backend service in most production deployments. Instead, they will create an API or proxy service and expose it for public use. By introducing this additional layer, we would be able to add some QoS for the service we are providing. We can consider authentication, usage metering, policy enforcement, and throttling as examples.

CreditService

The CreditService is the actual web service that does the crediting. It is called by the CreditProxy after getting the required information from the PersonInfoService. For this service, we should pass credit amount and all details about the user.

Credit Service API

API created by pointing to Credit Service hosted in WSO2 Application Server.

Setting up environment

The attached zip file contains all necessary configurations for the scenario. The scenario needs WSO2 API Manager and WSO2 Application Server. The files in the zipped folder should be copied or deployed into these servers. Now, let's look at the contents of the zip file and how to use them to create deployment with them. You could use this [1] link to download all resources for this example.

Now, let's look at the contents of the zip file and how to use them to create deployment with them.

Please use this[1] link to download all resources for this example.

Back-End services

Deploy the esb-samples-1.0-SNAPSHOT.jar as a jar service into the WSO2 Application Server. This jar has two POJO services. If you need to try this with your backend services, you can do that as well. The required classes that need to be exposed as web services are

org.wso2.esb.samples.CreditService

org.wso2.esb.samples.PersonInfoService

API Manager Configuration

1. Copy the contents of following folders in the sample zip to the repository/deployment/server/synapse-configs/default/ of WSO2 API manager prod. We can create these configurations using WSO2 ESB management console or developer studio. But to make it simple lets copy them directly and deploy to synapse environment. Following files need to deploy in API Manager.

/local­entries/xslt.xml

/proxy­services/CreditProxy.xml

/endpoints/CreditEpr.xml

/endpoints/PersonInfoEpr.xml

/sequences/creditSeq.xml

/sequences/personInfoSeq.xml

Here I have attached API configs as well. However, I recommend you to create two APIs for services using API publisher UI. Both of them should be pointed to actual services hosted in WSO2 Application Server. Follow these instructions to create APIs for this example.

2. Copy the personToCredit.xslt in the sample zip to wso2am-1.6.0/resources/ directory of WSO2 API Manager

3. Copy the CreditProxy.wsdl in the sample zip to the wso2am-1.6.0/resources/ directory of the WSO2 API Manager.

4. If you are running the WSO2 API Manager in the sample machine as the WSO2 Application Server, change the ports of the WSO2 API Manager in respository/conf/transport-mgt.xml. Otherwise there will be port conflicts.

Understand and test scenario

Now we have the completed configurations for the scenario; let's go through the API Manager configuration step by step to understand how it works. Here, we have one main entry point (credit Proxy or credit API). Receive a request with only the credit amount and ID of the requestor.

Send the ID to the PersonInfoService to get the Address and Name of the requestor. Use the credit amount, ID, address and name to create a request to the credit service and call the CreditService. The request to the proxy service is:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"xmlns:sam="http://samples.esb.wso2.org">
    <soapenv:Body>
        <sam:credit>
            <sam:id>99990000</sam:id>
            <sam:amount>1000</sam:amount>
        </sam:credit>
    </soapenv:Body>
</soapenv:Envelope>

As you see here we only need credit amount and user ID.

The information in the original request is required to call the two back­end services. When we do the first request, the information from the original request will be lost if we do not preserve it. Therefore, we need to store the required information from the original request in the message context. In this case, we are only going to store the ID and the credit amount in the message context. However, it is possible to store any part of the message in the context. For example, in some scenarios, the whole message is stored. We store the request ID and amount by extracting them and using the property mediator with XPath.

<property xmlns:sam="https://samples.esb.wso2.org" name="ORG_ID"
expression="//sam:credit/sam:id"/>
<property xmlns:sam="https://samples.esb.wso2.org" name="ORG_AMOUNT"
expression="//sam:credit/sam:amount"/>

After this step, the ID and the credit amount will be available in the Message Context properties named ORG_ID and ORG_AMOUNT, respectively. Now, we are ready to do the first request to the Person Info Service API. We need the XML for this request; therefore, we will insert it into the message using the Enrich Mediator.

<enrich>
    <source type="inline" clone="true">
        <sam:get xmlns:sam="https://samples.esb.wso2.org">
        <sam:id>?</sam:id>
        </sam:get>
    </source>
    <target type="body"/>
</enrich>

Then we will log message details and send request to PersonInfoEpr end point. Also please note that we have set receive sequence here. Then response will directly goto mentioned sequence(personInfoSeq) after completing initial service call. See following configuration.

Configuration to log Message

<log level="full">
<property name="sequence" value="inSequence ­ request for PersonInfoService"/>
</log>

Configuration to send request to PersonInfoEpr and receive response to personInfoSeq sequence.

<send receive="personInfoSeq">
<endpoint key="PersonInfoEpr"/>
</send>

Here you will see personInfoSeq configuration. There we have used xslt transformation to convert message to desired format for credit service. There you will noticed that response will direct to creditSeq. If you need to do some change to final response message you are free to do it there.

<sequence name="personInfoSeq">
<xslt key="xslt">
<property name="amount" expression="get­property('ORG_AMOUNT')"/>
</xslt>
 <property name="Action" value="urn:credit"/>
<send receive="creditSeq">
<endpoint key="CreditEpr"/>
</send>
</sequence>

Here is the creditSeq configuration. There we log message and send response to client directly.

<sequence name="creditSeq">
<log level="full"/>
<send/>
</sequence>

So this will cover the complete service chaining scenario using batch API. According to your specific scenarios, you might have to add more complex logics. We have tested this scenario with WSO2 API Manager 1.5.0 and WSO2 Application Server 5.2.1.

When you invoke this proxy service, create two APIs to credit service and personal information service. Then subscribe both of them to a single application. Thereafter, generate tokens for that application and invoke proxy service with that (for this example we have used proxy service if you need to create the API for that). Inside the proxy service, we will extract Auth headers and store it inside the message context; thereafter, we use that token to each and every following API calls. Refer to the the steps below:

01. create credit API by pointing to credit service

02. create personal information API by pointing to personal information service.

03. Subscribe all APIs to single application and generate tokens

Create credit service API by pointing to some dummy URL. We will add all mediation logic inside this and we don't need to point to specific end point. Later we will update and edit synapse configuration for this API. Go to synapse configuration source view and replace content of CreditServiceAPI.

Sometimes we need to edit synapse configuration manually after creating API from WSO2 API Manager publisher application. But please not that if you update same API using publisher application then changes made through synapse configuration editor will disappear. When mediation and processing become quite complex its always advisable to use API facade pattern. But for this specific example we will not use it.

We used following configuration to extract auth header and store it for following API calls. Then we will inject auth headers to each API call.

<property name="Auth" expression="get­property('transport','Authorization')"/>

We used above to extract auth header and store it for following API calls. This is how we inject auth headers for each API call.

If you enabled API manager wire logs you will see following logs for incoming and outgoing messages. When you try this sample always set your API urls to end points and use them inside your configuration. Then once you changed API with newer version no need to change urls in all the places.

[2014­02­05 15:38:45,775]  INFO ­ SequenceDeployer Sequence: main has been updated from the file:
/home/sanjeewa/work/workflow/wso2am­1.6.0­1/repository/deployment/server/synapse­configs/default/sequences/main.xml
[2014­02­05 15:38:46,684] DEBUG ­ wire >> "POST /services/CreditProxy.CreditProxyHttpSoap12Endpoint HTTP/1.1[\r][\n]"
[2014­02­05 15:38:46,684] DEBUG ­ wire >> "Content­Type: application/soap+xml; charset=UTF­8; action="urn:credit"[\r][\n]"
[2014­02­05 15:38:46,684] DEBUG ­ wire >> "Cookie: menuPanel=visible; menuPanelType=main;
region2_humantask_menu=visible; region2_bpel_instances_menu=visible; region3_registry_menu=none; i18next=en­US;
JSESSIONID=1579A304FBF0FE9AA2BEE6B9A7C9F85A;
requestedURI="../../carbon/service­mgt/index.jsp?region=region1&item=services_list_menu";
current­breadcrumb=manage_menu%2Cservices_menu%2Cservices_list_menu%23;
MSG13915816906030.6012256733392928=true; region1_configure_menu=none; region4_monitor_menu=none;
region5_tools_menu=none[\r][\n]"
[2014­02­05 15:38:46,684] DEBUG ­ wire >> "User­Agent: Axis2[\r][\n]"
[2014­02­05 15:38:46,684] DEBUG ­ wire >> "Host: sanjeewa­ThinkPad­T530:8280[\r][\n]"
[2014­02­05 15:38:46,685] DEBUG ­ wire >> "Transfer­Encoding: chunked[\r][\n]"
[2014­02­05 15:38:46,685] DEBUG ­ wire >> "[\r][\n]"
[2014­02­05 15:38:46,685] DEBUG ­ wire >> "130[\r][\n]"
[2014­02­05 15:38:46,685] DEBUG ­ wire >> "332[\r][\n]"
[2014­02­05 15:38:46,685] DEBUG ­ wire >> "0[\r][\n]"
[2014­02­05 15:38:46,685] DEBUG ­ wire >> "[\r][\n]"
[2014­02­05 15:38:46,688]  INFO ­ LogMediator To: /services/CreditProxy.CreditProxyHttpSoap12Endpoint, WSAction: urn:credit,
SOAPAction: urn:credit, MessageID: urn:uuid:4f418b82­d739­4cc8­bce1­f1fbd54659b3, Direction: request, sequence =
inSequence ­ request for CreditProxy, Envelope: 332
[2014­02­05 15:38:46,690]  INFO ­ LogMediator To: /services/CreditProxy.CreditProxyHttpSoap12Endpoint, WSAction: urn:credit,
SOAPAction: urn:credit, MessageID: urn:uuid:4f418b82­d739­4cc8­bce1­f1fbd54659b3, Direction: request, sequence =inSequence ­ request for PersonInfoService, Envelope: 
                            33
                        
[2014­02­05 15:38:46,691] DEBUG ­ wire << "POST /services/PersonInfoService/ HTTP/1.1[\r][\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "Cookie: menuPanel=visible; menuPanelType=main;
region2_humantask_menu=visible; region2_bpel_instances_menu=visible; region3_registry_menu=none; i18next=en­US;
JSESSIONID=1579A304FBF0FE9AA2BEE6B9A7C9F85A;
requestedURI="../../carbon/service­mgt/index.jsp?region=region1&item=services_list_menu";
current­breadcrumb=manage_menu%2Cservices_menu%2Cservices_list_menu%23;
MSG13915816906030.6012256733392928=true; region1_configure_menu=none; region4_monitor_menu=none;
region5_tools_menu=none[\r][\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "Content­Type: application/soap+xml; charset=UTF­8; action="urn:credit"[\r][\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "Transfer­Encoding: chunked[\r][\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "Host: localhost:9764[\r][\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "Connection: Keep­Alive[\r][\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "User­Agent: Synapse­PT­HttpComponents­NIO[\r][\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "[\r][\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "124[\r][\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "[\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "                            33[\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "                        [\r][\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "0[\r][\n]"
[2014­02­05 15:38:46,692] DEBUG ­ wire << "[\r][\n]"
[2014­02­05 15:38:46,695] DEBUG ­ wire >> "HTTP/1.1 200 OK[\r][\n]"
[2014­02­05 15:38:46,695] DEBUG ­ wire >> "Content­Type: application/soap+xml;charset=UTF­8[\r][\n]"
[2014­02­05 15:38:46,695] DEBUG ­ wire >> "Transfer­Encoding: chunked[\r][\n]"
[2014­02­05 15:38:46,695] DEBUG ­ wire >> "Date: Wed, 05 Feb 2014 10:08:46 GMT[\r][\n]"
[2014­02­05 15:38:46,695] DEBUG ­ wire >> "Server: WSO2 Carbon Server[\r][\n]"
[2014­02­05 15:38:46,695] DEBUG ­ wire >> "[\r][\n]"
[2014­02­05 15:38:46,695] DEBUG ­ wire >> "200[\r][\n]"
[2014­02­05 15:38:46,696] DEBUG ­ wire >> "59, Flower Road,
Colombo 07, Sri
Lanka33WSO2[\r][\n]"
[2014­02­05 15:38:46,697] DEBUG ­ wire >> "0[\r][\n]"
[2014­02­05 15:38:46,697] DEBUG ­ wire >> "[\r][\n]"
[2014­02­05 15:38:46,706] DEBUG ­ wire << "POST /services/CreditService/ HTTP/1.1[\r][\n]"
[2014­02­05 15:38:46,706] DEBUG ­ wire << "Content­Type: application/soap+xml;charset=UTF­8[\r][\n]"
[2014­02­05 15:38:46,707] DEBUG ­ wire << "Transfer­Encoding: chunked[\r][\n]"
[2014­02­05 15:38:46,707] DEBUG ­ wire << "Host: localhost:9764[\r][\n]"
[2014­02­05 15:38:46,707] DEBUG ­ wire << "Connection: Keep­Alive[\r][\n]"
[2014­02­05 15:38:46,707] DEBUG ­ wire << "User­Agent: Synapse­PT­HttpComponents­NIO[\r][\n]"
[2014­02­05 15:38:46,708] DEBUG ­ wire << "[\r][\n]"
[2014­02­05 15:38:46,708] DEBUG ­ wire << "200[\r][\n]"
[2014­02­05 15:38:46,708] DEBUG ­ wire << "[\n]"
[2014­02­05 15:38:46,708] DEBUG ­ wire << "[\n]"
[2014­02­05 15:38:46,708] DEBUG ­ wire << "2[\n]"[2014­02­05 15:38:46,708] DEBUG ­ wire << "[\n]"
[2014­02­05 15:38:46,709] DEBUG ­ wire << "59, Flower Road, Colombo 07, Sri Lanka[\n]"
[2014­02­05 15:38:46,709] DEBUG ­ wire << "33[\n]"
[2014­02­05 15:38:46,709] DEBUG ­ wire << "WSO2[\n]"
[2014­02­05 15:38:46,709] DEBUG ­ wire << "[\n]"
[2014­02­05 15:38:46,709] DEBUG ­ wire << "[\n]"
[2014­02­05 15:38:46,709] DEBUG ­ wire << "[\r][\n]"
[2014­02­05 15:38:46,709] DEBUG ­ wire << "0[\r][\n]"
[2014­02­05 15:38:46,710] DEBUG ­ wire << "[\r][\n]"
[2014­02­05 15:38:46,712] DEBUG ­ wire >> "HTTP/1.1 200 OK[\r][\n]"
[2014­02­05 15:38:46,712] DEBUG ­ wire >> "Content­Type: application/soap+xml;charset=UTF­8[\r][\n]"
[2014­02­05 15:38:46,712] DEBUG ­ wire >> "Transfer­Encoding: chunked[\r][\n]"
[2014­02­05 15:38:46,712] DEBUG ­ wire >> "Date: Wed, 05 Feb 2014 10:08:46 GMT[\r][\n]"
[2014­02­05 15:38:46,712] DEBUG ­ wire >> "Server: WSO2 Carbon Server[\r][\n]"
[2014­02­05 15:38:46,712] DEBUG ­ wire >> "[\r][\n]"
[2014­02­05 15:38:46,712] DEBUG ­ wire >> "109[\r][\n]"
[2014­02­05 15:38:46,713] DEBUG ­ wire >> "true[\r][\n]"
[2014­02­05 15:38:46,713] DEBUG ­ wire >> "0[\r][\n]"
[2014­02­05 15:38:46,713] DEBUG ­ wire >> "[\r][\n]"
[2014­02­05 15:38:46,714]  INFO ­ LogMediator To: https://www.w3.org/2005/08/addressing/anonymous, WSAction: ,
SOAPAction: , MessageID: urn:uuid:10f13946­23dc­49e0­97ab­a369c2101a9e, Direction: response, Envelope: true
[2014­02­05 15:38:46,717] DEBUG ­ wire << "HTTP/1.1 200 OK[\r][\n]"
[2014­02­05 15:38:46,717] DEBUG ­ wire << "Content­Type: application/soap+xml;charset=UTF­8[\r][\n]"
[2014­02­05 15:38:46,717] DEBUG ­ wire << "Date: Wed, 05 Feb 2014 10:08:46 GMT[\r][\n]"
[2014­02­05 15:38:46,717] DEBUG ­ wire << "Server: WSO2­PassThrough­HTTP[\r][\n]"
[2014­02­05 15:38:46,717] DEBUG ­ wire << "Transfer­Encoding: chunked[\r][\n]"
[2014­02­05 15:38:46,717] DEBUG ­ wire << "[\r][\n]"
[2014­02­05 15:38:46,718] DEBUG ­ wire << "109[\r][\n]"
[2014­02­05 15:38:46,718] DEBUG ­ wire << "true[\r][\n]"
[2014­02­05 15:38:46,718] DEBUG ­ wire << "0[\r][\n]"
[2014­02­05 15:38:46,718] DEBUG ­ wire << "[\r][\n]"
[2014­02­05 15:38:47,786]  INFO ­ SequenceDeployer Sequence: fault has been updated from the file:
/home/sanjeewa/work/workflow/wso2am­1.6.0­1/repository/deployment/server/synapse­configs/default/sequences/fault.xml

Resources

Download resources required for this sample

References

[1] https://drive.google.com/file/d/0B3OmQJfm2Ft8ZWRxRlUxd1VTeXM/edit?usp=sharing

[2] https://wso2.com/products/api-manager/

[2] https://docs.wso2.com/display/AM160/Creating+an+API

Conclusion

API chaining pattern becomes useful when you need to implement service chaining with APIs. With this approach, you will be able to minimize the number of service calls between the client and the server. However, users need to be aware of synapse configurations to configure batch APIs. In summary, this would be ideal for mobile applications and clients because you could optimize network usage. Moreover, it minimizes the complexity of client application logic as well.