2012/09/10
10 Sep, 2012

How to GET a Cup of Coffee the WSO2 Way

  • Hiranya Jayathilaka
  • PhD student - WSO2

Applies To

Contents

REST Support in WSO2 Middleware

Before we dive into the implementation details of our solution lets take a few minutes to review and understand the level of REST support offered by WSO2 middleware. We are mainly interested in the REST support provided by WSO2 Application Server (AS) and WSO2 Enterprise Service Bus (ESB).

WSO2 AS is a runtime for deploying Java web services and web applications. It is based on a number of well known open source projects such as Apache Axis2 and Apache Tomcat. The underlying Axis2 engine of WSO2 AS generates an HTTP binding for all the deployed web services. Therefore any web service deployed on WSO2 AS can be invoked by making pure HTTP/XML calls without using SOAP as an intermediate protocol. For an example consider the following service implementation.

package com.wso2.samples;

import java.util.Random;

public class MathService {
	
	public int getRandomNumber() {
		Random rand = new Random();
		return rand.nextInt();
	}
	
	public int add(int a, int b) {
		return a + b;
	}

}

This service contains 2 operations, namely “getRandomNumber” and “add”. Once this service is deployed in WSO2 AS, service consumers will be able to invoke the “getRandomNumber” operation by making a simple HTTP GET call as follows:

GET /services/MathService/getRandomNumber HTTP/1.1

The “add” operation takes 2 integers as input parameters. Consumers can invoke this operation by making a HTTP GET call where the arguments are encoded as URL query parameters or by making a HTTP POST where the inputs are embedded in a simple XML body.

GET /services/MathService/add?a=5&b=10
POST /services/MathService HTTP/1.1
Host: 127.0.0.1:9763
Accept: */*
Content-Type: application/xml
Content-Length: 80

<p:add xmlns:p="https://samples.wso2.com">   
   <p:a>5</p:a>   
   <p:b>10</p:b>
</p:add>

One may also leverage the support for web applications provided by WSO2 AS to deploy JAX-RS based pure REST applications. A JAX-RS application developed using a framework such as Apache Wink or CXF can be easily deployed on WSO2 AS. Starting from the next major release of WSO2 AS, it will have first class support for JAX-RS. This will make developing, packaging and deploying JAX-RS based RESTful applications in WSO2 AS even easier.

WSO2 ESB can be used to route, filter and transform RESTful invocations. Any proxy service or sequence deployed in WSO2 ESB can receive and process REST calls. It provides a convenient approach for interfacing RESTful clients with SOAP services and SOAP clients with RESTful services. Starting from version 4.0.3, WSO2 ESB also has comprehensive support for exposing REST APIs. Much of the implementation work described in this article revolves around this new feature of WSO2 ESB. Therefore let’s take a closer look at the REST API support in WSO2 ESB.

REST API Support in WSO2 ESB

A REST API in WSO2 ESB is analogous to a web application deployed in the ESB runtime. Each API is anchored at a user defined URL context, much like how a web application deployed in a servlet container is anchored at a fixed URL context. An API will only process requests that fall under its URL context. For an example if a particular API is anchored at the context “/test”, only those HTTP requests whose URL path starts with “/test” will be handled by that API. It is also possible to bind a given API to a user defined hostname and/or a port number.

A REST API is made of one or more resources. From an architectural standpoint, a resource is a logical component of an API which can be accessed by making a particular type of HTTP calls. From a more pragmatic angle, a resource is similar to a proxy service. Just like a proxy service, a resource also contains an in-sequence, an out-sequence and a fault-sequence. But there are also several significant differences between resources and proxy services. For instance a resource can only be used to receive and process REST calls whereas a proxy service can receive all types of requests. Also a resource cannot publish a WSDL nor other WS-* modules (security, reliable messaging etc) can be engaged on a resource.

A resource can be associated with a user defined URL pattern or a URI template. This way we can restrict the type of HTTP requests processed by a particular resource. In addition to that a resource can be bound to a specific subset of HTTP verbs and header values. This option provides additional control over what requests are handled by a given resource. For an example consider a resource associated with the URL pattern “/foo/*” and the HTTP verb “GET”. This will make sure that the resource will only process GET requests whose URL path matches the pattern “/foo/*”. Therefore following requests will be processed and mediated by the resource:

GET /test/foo/bar
GET /test/foo/a?arg1=hello

Following HTTP requests will not be handled by the above mentioned resource.

GET /test/food/bar (URL pattern not matching)
POST /test/foo/bar (HTTP verb not matching)

Once a request is dispatched into a resource it will be mediated through the in-sequence of the resource. At the end of the in-sequence the request can be forwarded to a back-end application for further processing. Any responses coming from the back-end system are mediated through the out-sequence of the resource. The fault-sequence is used to handle any errors that may occur while mediating a message through a resource.

To further understand how request dispatching works among APIs and resources, let’s consider the following object hierarchy.

Here we have two REST APIs anchored at contexts “/foo” and “/bar” respectively. Each API is made of two resources. A GET request to the path “/foo/test” will be received by the first API since it's anchored at context “/foo”. Then it will be handed to resource A which is configured to process GET requests with the URL pattern “/test/*”. Similarly a POST request to the path “/foo/test” will be received by resource B in the same API. PUT requests to the path “/bar/test” will be dispatched to resource C in the second API and DELETE requests to the path “/bar/abc” will be processed by resource D.

The above example demonstrates the usefulness of the REST API feature in WSO2 ESB. In a nutshell, it provides a simple yet very powerful approach for breaking down a stream of HTTP calls based on HTTP methods, URL patterns and various other parameters. Once the HTTP calls have been filtered out and dispatched to appropriate APIs and resources, we can subject them to the routing and mediation capabilities of the ESB using mediators, sequences and endpoints.

URL Mappings and URI Templates

As stated earlier, a resource can be associated with a URL mapping or a URI template. A URL mapping could be any valid servlet mapping. Hence, as stated in the servlet specification, there are three types of URL mappings:

  • Path mappings (eg: /test/*, /foo/bar/*)
  • Extension mappings (eg: *.jsp, *.do)
  • Exact mappings (eg: /test, /test/foo)

When a resource is defined with a URL mapping, only those requests that match the given URL mapping will be processed by the resource. Alternatively one could configure a resource with a URI template. A URI template represents a class of URIs using patterns and variables. Some examples of valid URI templates are given below.

/order/{orderId}
/dictionary/{char}/{word}

All the identifiers within curly braces are considered variables. A URL that matches the template “/order/{orderId}” is given below.

/order/A0001

In the above URL instance, the variable orderId has been assigned the value “A0001”. Similarly following URL adheres to the template “/dictionary/{char}/{word}”.

/dictionary/c/cat

In this case the variable “char” has the value “c” and the variable “word” is given the value “cat”. When a resource is associated with a URI template, all requests that match the template will be processed by the resource. At the same time ESB will provide access to the exact values of the template variables through message context properties. For an example assume a resource configured with the URI template “/dictionary/{char}/{word}”. If the request “/dictionary/c/cat” is sent to the ESB, it will be dispatched to the above resource and we will be able to retrieve the exact values of the two variables using the get-property XPath extension of WSO2 ESB:

<log level="custom">
    <property name="Character" expression="get-property('uri.var.char')"/>
    <property name="Word" expression="get-property('uri.var.word')"/>
</log>

Above log mediator configuration would generate the following output for the request “/dictionary/c/cat”.

Configuring APIs and Resources in ESB

Now that we have a basic understanding of how REST APIs work in WSO2 ESB, let’s take a quick look at how we can configure an API in the service bus using the ESB configuration language.

An API definition is identified by the <api> tag. Each API must specify a unique name and a unique URL context. One or more tags can be enclosed within an API definition. Some example API definitions are given below.

<api name="API_1" context="/order">
    <resource url-mapping="/list" inSequence="seq1" outSequence="seq2"/>
</api>
<api name="API_2" context="/user">
    <resource url-mapping="/list" methods="GET" inSequence="seq3" outSequence="seq4"/>
    <resource uri-template="/edit/{userId}" methods="PUT POST" inSequence="seq5" outSequence="seq6"/>
</api>
<api name="API_3" context="/payments">
    <resource url-mapping="/list" methods="GET" inSequence="seq7" outSequence="seq8"/>
    <resource uri-template="/edit/{userId}" methods="PUT POST" outSequence="seq9">
        <inSequence>
             <log/>
             <send>
                  <endpoint key="BackendService"/>
             </send>
        </inSequence>
    </resource>
    <resource inSequence="seq10" outSequence="seq11"/>
</api>

Note the last resource definition in API_3 which does not specify a URL mapping nor a URI template. This is called the default resource of the API. Each API can have at most one default resource. Any request received by the API but does not match any of the enclosed resource definitions will be dispatched to the default resource of the API. In case of API_3, a DELETE request on the URL “/payments” will be dispatched to the default resource as none of the other resources in API_3 are configured to handle DELETE requests.

Designing the Starbucks Solution

Now we have enough background knowledge on WSO2 AS and WSO2 ESB to get started. As mentioned earlier we are going to implement the RESTful solution described in Jim Webber's article. If you haven't read this article yet, it's highly recommended that you take some time to go through it before attempting the rest of this article.

The Starbucks application described in Webber's article consists of two distinct state machines:

  • Customer state machine
  • Barista state machine

The customer state machine consists of following application interactions:

  • Customer placing a new order for a drink
  • Customer making changes to an already placed order (eg: requesting a specific flavor to be added)
  • Customer making the payment for the order

On the other hand, the barista state machine is made up of following application interactions:

  • Retrieving a list of all pending orders
  • Checking whether the payment has been received for a particular order
  • Removing delivered orders from the list of pending orders

Based on the above details, we can identify three main applications involved in this solution.

  • Customer application
  • Barista application
  • Starbucks OMS (Order Management System)

The customer application would interact with the Starbucks OMS to place orders and make payments. Similarly the barista application would interact with the Starbucks OMS to obtain the list of pending orders and check payment status of individual orders. With that in mind we can come up with a high-level solution architecture for the Starbucks coffee house scenario as follows.

When it comes to implementing the above solution we are not going to pay too much attention to the implementation of 3 applications. We are mainly interested in how the applications interact with each other using a RESTful communication model. For the Starbucks OMS we are going to use a simple Java web service which we will host in WSO2 AS. This is going to be a SOAP service which will store all the order details in an in-memory table. If this was an application to be used in a real-world scenario, it would be much more complex with probably one or more databases. To simulate the customer application and the barista application we can use a simple HTTP client tool like Curl. Towards the latter part of this article we will look at a couple of simple Java GUI applications which can be used to simulate the customer and barista behavior.

Because our Starbucks OMS is based on SOAP we are going to have to use some sort of a REST to SOAP converter. For this purpose we are going to use WSO2 ESB. We will expose a set of REST APIs in the ESB. The customer and barista applications would interact with these APIs by making RESTful invocations. ESB will translate the REST calls into SOAP interactions and invoke the Starbucks OMS on WSO2 AS. With this information we can further improve our solution architecture diagram as follows.

In addition to RESTful system design, the above solution teaches us how to expose an existing system over REST. Basically we take the Starbucks OMS which is based on SOAP and implement a RESTful overlay on top it. The same technique can be further extended to expose any application over REST. Many organizations have services and applications that need to be exposed over REST so they can be easily consumed by web browsers and mobile applications. Information discussed in this article provides the basis for putting such a use case into action.

Setting up the Servers

We will start by installing WSO2 AS and WSO2 ESB. Recommended product versions for this exercise are WSO2 AS 4.1.2 and WSO2 ESB 4.0.3. API support was first introduced in WSO2 ESB 4.0.3 so older versions of the ESB cannot be used for the purpose of this tutorial.

To install, simply extract the downloaded zip distributions. Open up the repository/conf/carbon.xml file of the ESB and locate the “Ports” configuration. Change the “Offset” setting from 0 to 1:

<Ports>
    <Offset>1</Offset>
    ...
</Ports>

This will increment all the ports used by the ESB by 1. Without this change both ESB and AS will try to listen on the same ports and will run into bind exceptions. Once the above change is done in the ESB, start both AS and ESB by executing the wso2server startup script in the bin directory of each installation.

We will also make this an opportunity to deploy and setup the Starbucks OMS. Simply login to the AS management console at https://localhost:9443/carbon. Click on the “Web Services > Add > Axis2 Service” option in the “Manage” menu. Now download the StarbucksOutletService.aar file and upload it to AS (source code of this web service can be found here).

It will take a few seconds for the service to get deployed properly. You can track the progress of the process by tracing the AS server log.

[2012-08-15 18:16:13,177]  INFO {org.wso2.carbon.core.deployment.DeploymentInterceptor} -  Deploying Axis2 service: StarbucksOutletService {super-tenant}
[2012-08-15 18:16:13,635]  INFO {org.apache.axis2.deployment.DeploymentEngine} -  Deploying Web service: StarbucksOutletService.aar - file:/home/hiranya/Desktop/starbucks/wso2as-4.1.2/repository/deployment/server/axis2services/StarbucksOutletService.aar

Once the service has been properly deployed, click on the “Web Services > List” option. You will see that a new service named “StarbucksOutletService” has been deployed on the server.

You can use a traditional SOAP client like SOAP UI to interact with this service. Alternatively you can use the “Try-It” option provided by AS to invoke the service and try it out.

Placing New Orders

Let’s start by implementing the REST API which is responsible for accepting new orders. The back-end Starbucks OMS provides an addOrder operation to handle this task. So our REST API should ultimately invoke that operation to get the job done. We can name our API “StarbucksOrderAPI” and anchor it at "/order" context as follows.

<api name="StarbucksOrderAPI" context="/order">
  ...
</api>

According to Webber's article, a new order is submitted by sending a HTTP POST request to the “/order” URL context. So we need to define a resource in our new API to accept such POST requests.

<api name="StarbucksOrderAPI" context="/order">
   <resource url-mapping="/" methods="POST">
      …
   </resource>
</api>

Now we can go ahead and implement the in-sequence and out-sequence for the above resource. The in-sequence should construct the SOAP payload expected by the addOrder operation in Starbucks OMS and invoke that operation.

<inSequence>
        <property name="STARBUCKS_HOST_NAME" expression="$axis2:SERVICE_PREFIX" />
        <payloadFactory>
                <format>
                        <m0:addOrder>
                                <m0:drinkName>$1</m0:drinkName>
                                <m0:additions>$2</m0:additions>
                        </m0:addOrder>
                </format>
                <args>
                        <arg expression="//sb:drink" />
                        <arg expression="//sb:additions" />
                </args>
        </payloadFactory>
        <send>
                <endpoint key="DataServiceEndpoint" />
        </send>
</inSequence>

The out-sequence should transform the SOAP response into an acceptable RESTful payload format, set the appropriate status code and pass the message on to the client.

<outSequence>
        <property name="HTTP_SC" value="201" scope="axis2" />
        <property name="uri.var.orderId" expression="//m1:orderId"/>
        <sequence key="StarbucksOrderInfo" />
        <send />
</outSequence>

The complete API configuration for the Starbucks sample is available in the synapse-configs.zip file. You can simply extract this into the repository/deployment/server directory of your ESB installation and replace the existing configuration before starting the server. Once properly deployed, invoke the StarbucksOrderAPI and try it out. If you are using Curl, put the following XML snippet into a file named order.xml

<?xml version="1.0" encoding="UTF-8"?>
<order xmlns="https://starbucks.example.org">
  <drink>Caffe Misto</drink>
</order>

Then execute the following Curl command.

curl -v -d @order.xml -H "Content-type: application/xml" https://localhost:8281/order

The request sent by the client (customer application) and the response sent by the ESB are as follows.

POST /order HTTP/1.1
Transfer-Encoding: chunked
Content-Type: application/xml
Host: 127.0.0.1:8281
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.2 (java 1.5)

<?xml version="1.0" encoding="UTF-8"?>
<order xmlns="https://starbucks.example.org">
  <drink>Caffe Misto</drink>
</order>
HTTP/1.1 201 Created
Content-Type: application/xml; charset=UTF-8
Location: https://127.0.0.1:8281/order/7b36032d-6aa2-4d77-90a9-7473a42de77b
Server: WSO2 Carbon Server
Vary: Accept-Encoding
Date: Wed, 15 Aug 2012 12:55:50 GMT
Transfer-Encoding: chunked
Connection: Keep-Alive

<?xml version="1.0" encoding="UTF-8"?>
<order xmlns="https://starbucks.example.org">
  <drink>Caffe Misto</drink>
  <cost>6.99</cost>
  <additions/>
  <next rel="https://127.0.0.1:8281/payment" type="application/xml"
    uri="https://127.0.0.1:8281/payment/order/7b36032d-6aa2-4d77-90a9-7473a42de77b" xmlns="https://example.org/state-machine"/>
</order>

Let's walk through the API configuration and try to understand how it works. The first thing to note is the following property mediator instance used in the in-sequence.

<property name="STARBUCKS_HOST_NAME" expression="$axis2:SERVICE_PREFIX" />

“SERVICE_PREFIX” is a property available in ESB messages flows by default. Set in the “axis2” scope, this property contains the [protocol]://[host]:[port] segment of the URL that was invoked by the client. For instance if the user sent a request to the URL https://localhost:8280/foo/bar, then the above property will contain the value https://localhost:8280/. Using the above property mediator configuration we assign this value to a new property named “STARBUCKS_HOST_NAME”. We intend to use this value later when formulating the response which should be sent to the client.

The property mediator in the in-sequence is followed by a payload factory mediator. This mediator is used to construct the SOAP payload that needs to be sent to Starbucks OMS. We execute a few XPath expressions against the original request to extract some of its values and put them in the newly constructed addOrder payload.

At the end of the in-sequence we have used a send mediator with an endpoint. This is where we call into the Starbucks OMS. The actual endpoint is defined as follows:

<endpoint name="DataServiceEndpoint" xmlns="http://ws.apache.org/ns/synapse">
        <address uri="https://localhost:9763/services/StarbucksOutletService" format="soap12"/>
</endpoint>

Note the format=”soap12” attribute. This tells the ESB that all messages sent to the OMS endpoint should be in SOAP 1.2 format. Therefore the ESB will wrap the XML payload in a SOAP 1.2 envelope before sending it to the back-end service.

If everything works out fine, ESB will receive the following SOAP response from AS. Note how the OMS has generated a unique ID for the newly created order. This SOAP message will be mediated through the out-sequence of the appropriate API resource. In our out-sequence we have the following property mediator instance as the first entry.

<property name="HTTP_SC" value="201" scope="axis2" />

The above property mediator changes the HTTP status code of the response. According to proper RESTful design principles we should send a 201 Created response for this scenario. But what we get from the Starbucks OMS is a 200 OK response (typical SOAP endpoint behavior). The above configuration will change that to 201 Created so the client receives the correct response.

In a proper RESTful design it is also crucial to send a Location header along with a 201 Created response. This header should contain a valid URL which points back to the resource that was created in the back-end system. Therefore in our scenario the Location header should contain a URL by which we can obtain a description of the order that we just submitted. Following property mediator configuration in the “StarbucksOrderInfo” sequence is used to add this Location header to the outgoing response.

<property name="Location"
                expression="concat($ctx:STARBUCKS_HOST_NAME, 'order/', //m1:orderId)"
                scope="transport" />

As you can see this is where we make use of the STARBUCKS_HOST_NAME property that we initialized earlier in the in-sequence. We simply take the value of this property and append the “/order/orderId” fragment to construct the complete URL.

Constructing responses that contain URLs of various related resources is paramount in REST application architecture. This is particularly useful in establishing easy navigation and smooth state transitions within the application. The rationale is that the response of a RESTful invocation should contain all the information a client would require to navigate or proceed to the next step within the application. That way a client can start with a single well-known URL and navigate the entire application by following the URLs included in each response. This concept is sometimes also referred to as HATEOAS (Hypermedia as the Engine of Application State). Note how we have incorporated this feature into our solution using the mediation capabilities of WSO2 ESB. The response to the order creation request contains a Location header which points back to the order resource. Also the response payload contains more URLs using which order payments can be carried out. Using the SERVICE_PREFIX built-in property we construct all the URLs in a manner so that they all point back to the ESB. At no point we expose the endpoint details of the backend web service.

Reviewing Orders

Once an order has been placed in the system, the customer should be able to review it. This is done by sending a HTTP GET request to the URL specified in the 201 Created responses. In our API configuration we have defined a separate resource to process these GET requests.

<resource uri-template="/{orderId}" methods="GET PUT OPTIONS" faultSequence="StarbuckFault">

Note how we have used a URI template to specify the incoming request URL format. The order ID part of the URL has been specified as a variable (orderId) because each order will have its own unique identifier. In the in-sequence we construct a getOrder SOAP payload using the payload factory mediator. There we utilize the value of the orderId variable.

<payloadFactory>
        <format>
                <m0:getOrder>
                        <m0:orderId>$1</m0:orderId>
                </m0:getOrder>
        </format>
        <args>
                <arg expression="$ctx:uri.var.orderId" />
        </args>
</payloadFactory>

The response from the Starbucks OMS will be a SOAP payload containing order details. We simply convert it into a plain old XML (POX) document and send back to the client as a RESTful response. To try this scenario out, find out the unique ID of the order you submitted earlier (you can get this from the Location header of the response to the order submission request) and run Curl as follows (replace my-order-id with the actual order ID):

curl -v https://localhost:8281/order/my-order-id

This is a good place to demonstrate some of the error handling capabilities of the ESB as well. Try invoking Curl as follows with an invalid order ID string.

curl -v https://localhost:8281/order/bogus-order-id

In this case the Starbucks OMS will return an empty payload to the ESB. The out-sequence of the corresponding ESB resource has been configured to detect this condition and respond with a HTTP 404 Not Found response. Therefore if you try the above Curl command, you will get an output similar to the following.

HTTP/1.1 404 Not Found
Content-Type: application/xml; charset=UTF-8
Server: WSO2 Carbon Server
Date: Wed, 15 Aug 2012 13:10:10 GMT
Transfer-Encoding: chunked

<message xmlns="https://starbucks.example.org"><text>No order exists by the specified ID</text></message>

Making Payments

We are going to skip ahead a couple of steps and see how payments can be handled in our solution. In Webber’s article, payments are treated as a separate type of resources. Therefore making a payment is equivalent to creating a new payment resource. Once a payment resource has been created, the user and the barista should be able to read the payment resources too.

In our implementation we handle payments through a separate API. Let’s call this the StarbucksPaymentAPI and anchor it at the context “/payment”. The customer can create a payment resource by sending a PUT request to the URL https://localhost:8281/payment/order/orderId. The payload of the request should contain all the required payment information such as amount and credit card details.

<resource uri-template="/order/{orderId}" methods="GET PUT" faultSequence="StarbucksFault">

Note the resource configured to handle PUT requests. This will transform the incoming request into a SOAP payload and invoke the Starbucks OMS. The response will be transformed back to a POX message and sent to the client as a 201 Created response. As usual a Location header which points back to the payment resource will be added to the response.

To try this out add the following XML payload to a file named payment.xml.

<?xml version="1.0" encoding="UTF-8"?>
<payment xmlns="https://starbucks.example.org">
  <cardNo>1234-5678-9010</cardNo>
  <expires>12/15</expires>
  <name>Peter Parker</name>
  <amount>6.99</amount>
</payment>

Now execute the following Curl command (replace order-id with an actual order ID).

curl -v -X PUT -d @payment.xml -H "Content-type: application/xml" https://localhost:8281/payment/order/order-id

You should receive a response similar to the following.

HTTP/1.1 201 Created
Content-Type: application/xml; charset=UTF-8
Location: https://127.0.0.1:8281/payment/order/090abb1b-9da0-4eb3-86c1-02c7fa157514
Server: WSO2 Carbon Server
Date: Wed, 15 Aug 2012 13:12:15 GMT
Transfer-Encoding: chunked
Connection: Keep-Alive

<?xml version="1.0" encoding="UTF-8"?>
<payment xmlns="https://starbucks.example.org/">
  <cardNo>1234-5678-9010</cardNo>
  <expires>12/15</expires>
  <name>Peter Parker</name>
  <amount>6.99</amount>
</payment>

Once a payment has been made, the customer can review the payment details by making a GET request. Find the unique ID of the payment resource and execute the following Curl command (replace order-id with the actual ID of the payment resource you want to review).

curl -v https://localhost:8281/payment/order/order-id
HTTP/1.1 200 OK
Content-Type: application/xml; charset=UTF-8
Server: WSO2 Carbon Server
Date: Wed, 15 Aug 2012 13:17:39 GMT
Transfer-Encoding: chunked
Connection: Keep-Alive

<?xml version="1.0" encoding="UTF-8"?>
<payment xmlns="https://starbucks.example.org/">
  <cardNo>1234-5678-9010</cardNo>
  <expires>12/15</expires>
  <name>Peter Parker</name>
  <amount>6.99</amount>
</payment>

The Starbucks barista can use the above feature to check whether an order has been paid for before the drink is released to the customer. If the payment hasn’t been made for a particular order, the API will return a 404 Not Found response.

Handling Order Updates

Once the customer has submitted an order, he should be able to make amendments to it. However the customer must make all the updates before the barista starts making the drink. Once the barista has prepared the drink, the customer shouldn’t be able to make changes. Therefore when supporting this requirement in our solution, we should make it possible for the customer to know whether the barista has started preparing an order or not.

Jim Webber’s article suggests using the HTTP OPTIONS verb as a means of checking whether an order is modifiable or not. A simple OPTIONS request to an order resource would return a response with the HTTP Allow header. If the order is modifiable the Allow header should have both “GET” and “PUT” values. Otherwise it will only have the value “GET” which means the order is read-only.

Supporting this requirement in our implementation is easy. Simply add the “OPTIONS” verb to the list of methods supported by the get-order resource.

<resource uri-template="/{orderId}" methods="GET PUT OPTIONS" faultSequence="StarbuckFault">

A simple switch case can further filter out the OPTIONS requests within the resource.

<switch source="$ctx:REST_METHOD">
	<case regex="OPTIONS">
		<property name="NO_ENTITY_BODY" value="true" scope="axis2" type="BOOLEAN" />
		<filter source="//m1:locked" regex="false">
		        <then>
		                <property name="Allow" value="GET,PUT" scope="transport" />
		        </then>
		        <else>
		                <property name="Allow" value="GET" scope="transport" />
		        </else>
		</filter>
	</case>
	...
</switch>

The resource can then consult the Starbucks OMS to check whether the order is modifiable or not. Depending on the response from the OMS, a suitable HTTP response can be formulated in the ESB.

To try this out, invoke Curl on an order resource as follows (replace order-id with an actual order ID).

curl -v -X OPTIONS https://localhost:8281/order/order-id

If the order is modifiable, you will see a response similar to the following.

HTTP/1.1 200 OK
Allow: GET,PUT
Server: WSO2 Carbon Server
Date: Wed, 15 Aug 2012 13:22:36 GMT
Content-Length: 0
Connection: Keep-Alive

Actual order updates are carried out by making a HTTP PUT call. The payload of the request should contain an updated description of the order. We have configured our API resource in the ESB to handle PUT requests as follows.

<resource uri-template="/{orderId}" methods="GET PUT OPTIONS" faultSequence="StarbuckFault">

Try this out by putting the following XML payload to a file named update.xml and executing the given Curl command (replace order-id with the actual order ID).

<?xml version="1.0" encoding="UTF-8"?>
<order xmlns="https://starbucks.example.org">
  <drink>Caffe Misto</drink>
  <additions>Milk</additions>
</order>
curl -v -X PUT -d @update.xml -H "Content-type: application/xml" https://localhost:8281/order/order-id

If all goes well, you will receive a 200 OK response confirming the update operation.

HTTP/1.1 200 OK
Content-Type: application/xml; charset=UTF-8
Location: https://127.0.0.1:8281/order/764cc8b5-e612-47e1-818b-225cb35b0c3f
Server: WSO2 Carbon Server
Date: Wed, 15 Aug 2012 13:24:48 GMT
Transfer-Encoding: chunked
Connection: Keep-Alive

<?xml version="1.0" encoding="UTF-8"?>
<order xmlns="https://starbucks.example.org">
  <drink>Caffe Misto</drink>
  <cost>10.71</cost>
  <additions>Milk</additions>
  <next rel="https://127.0.0.1:8281/payment" type="application/xml"
    uri="https://127.0.0.1:8281/payment/order/764cc8b5-e612-47e1-818b-225cb35b0c3f" xmlns="https://example.org/state-machine"/>
</order>

Retrieving the List of Pending Orders

At this point we are finished implementing all the interactions between the customer and the Starbucks OMS. Customers can now place drink orders, review and update them and also make payments. So it’s time to implement the interactions between the barista and the OMS. The very first interaction we are going to implement is the retrieval of pending order list. The OMS web service provides an operation named getOrders to support this use case. However in the solution suggested by Jim Webber, the barista should be able to access the list of orders in the form of a live Atom feed. So in our case it’s up to the ESB to make the transformation from SOAP to Atom.

We start by defining a new API named “StarbucksOrderListAPI”. This API is anchored at the context “/orders”. It consists of a single resource which handles GET requests.

<api name="StarbucksOrderListAPI" context="/orders">
        <resource methods="GET" faultSequence="StarbucksFault">
        ...
        </resource>
</api>

The resource will simply contact the back-end OMS service to retrieve the list of pending orders in SOAP format. Then in the out-sequence we apply an XSLT transformation to convert the SOAP response into a valid Atom feed.

<xslt key="OrderFeedGenerator">
        <property name="SystemDate" expression='get-property("SYSTEM_DATE", "yyyy-MM-dd'T'hh:mm:ss'Z'")'/>
        <property name="SystemURL" expression="$ctx:STARBUCKS_SYSTEM_URL"/>
</xslt>

However changing the payload format is not enough. Unless we send a proper content-type header with the response, the calling HTTP client would not recognize that the response is an Atom feed. Therefore we have to add the following property mediator to the configuration which will take care of sending the response back with the “application/atom+xml” content type.

<property name="ContentType" value="application/atom+xml" scope="axis2"/>

Now to try this API out send a GET request from Curl as follows.

curl -v https://localhost:8281/orders

If you have submitted any orders before through the StarbucksOrderAPI, you will receive an Atom feed similar to the following as a response. Some web browsers like Internet Explorer have built-in Atom feed readers. So if you access the above API using such a browser, you will see a nicely formatted output as follows.

Checking Payment Status

Before the barista can deliver a drink, he should check whether the customer has paid for the drink. Our StarbucksPaymentAPI already supports this. If the barista knows a particular order ID, he can check the payment status of the order by sending a HTTP GET request to the corresponding payment resource.

curl -v https://localhost:8281/payment/order/order-id

The above will return a 200 OK response along with a payment description, if the client has made the payment. Otherwise it will return 404 Not Found.

Removing Completed Orders from the List

Barista should remove prepared orders from the pending order list, so that the same order does not get processed multiple times. The back-end OMS has a removeOrder operation which can be used to implement this scenario. In a RESTful design, the proper way to remove a resource is by sending a HTTP DELETE request. So we define the following resource in the StarbucksBaristaAPI.

<resource uri-template="/order/{orderId}" methods="PUT DELETE">

This resource will accept DELETE requests, invoke the removeOrder operation in the OMS and remove the specified orders from the list. Note the use of the URI template “/orders/{orderId}”. This way barista can invoke the same resource by specifying different order ID values. In the in-sequence of the resource we construct a removeOrder payload using the value of the orderId template variable.

To try this action out, execute the following Curl command. Replace order-id with an actual order ID string.

curl -v -X DELETE https://localhost:8281/barista/order/order-id

You should get a response similar to the following.

HTTP/1.1 200 OK
Content-Type: application/xml; charset=UTF-8
Server: WSO2 Carbon Server
Date: Wed, 15 Aug 2012 13:38:20 GMT
Transfer-Encoding: chunked
Connection: Keep-Alive

<?xml version="1.0" encoding="UTF-8"?>
<message xmlns="https://starbucks.example.org">Order deleted</message>

To ensure that the order has been removed from the list, send a GET request to StarbucksOrderListAPI and check the returned Atom feed.

Improving the Overall Solution

By now we have a fairly complete, working solution at hand. Both workflows described in Jim Webber’s article have been implemented satisfactorily. However there’s plenty of room for improvement. In this section we are going to look at how certain non-functional requirements such as security, usability and performance can be incorporated into the solution we just implemented.

Content Negotiation

Content negotiation is a mechanism by which a client and a server communicate with each other and decide on a content format to use for data transfer. In our solution so far, we used XML (application/xml) as the primary means of data transfer. However we could use other content formats such as plain text and JSON to achieve the same result.

Real world client applications usually have their own preferred content types. For an example a web browser would usually prefer HTML. A Java based desktop application is likely to prefer POX. A mobile application would usually prefer JSON. In order to maintain interoperability, the server side applications should be prepared to serve content using any of these formats. Using content negotiation the client can indicate its content type preferences to the server, and the server can serve the requests using a content type preferred by the client.

HTTP specification provides the basic elements to build a powerful content negotiation framework. A HTTP client can indicate its content type preferences by sending the Accept header along with the requests. The client can indicate zero, one or more preferred content types. When no preferred content type is specified, the server should default to one of the supported content types. If the client wants to specify multiple preferences, the HTTP specification allows associating a priority value with each preference.

Let’s see how we can add some basic content negotiation support to our solution. In this example we are not going to pay attention to any priority values sent by the client. Rather we are going to have our own priority order for handling multiple preferences. We’ll use the StarbucksOrderListAPI as our guinea pig. First of all we need to retrieve the value of the Accept header sent by the client. We do this in the in-sequence as follows.

<property name="STARBUCKS_ACCEPT" expression="$trp:Accept"/>

Now in the out-sequence we can run a few comparisons to see which content type to use for sending the response.

<switch source="$ctx:STARBUCKS_ACCEPT">
        <case regex=".*atom.*">
                ...
        </case>
        <case regex=".*text/html.*">
                ...
        </case>
        <case regex=".*json.*">
                ...
        </case>
        <case regex=".*application/xml.*">
                ...
        </case>
        <default>
                ...
        </default>
</switch>

The above configuration results in the following priority order of content types.

  1. Atom (Also used as default)
  2. HTML
  3. JSON
  4. POX

To try this out, invoke the following Curl commands and see how the response format changes according to the value of the Accept header.

curl -v https://localhost:8281/orders
curl -v -H "Accept: application/xml" https://localhost:8281/orders
curl -v -H "Accept: application/json" https://localhost:8281/orders
curl -v -H "Accept: text/html" https://localhost:8281/orders

Also try accessing the same API from several Web browsers. Most web browsers send the Accept: text/html header with the requests. Therefore if you try to access the above URL from a browser like FireFox or Chrome, you will get a nicely formatted HTML page in return. Internet Explorer however doesn’t seem to send the Accept header. As a result IE will render the page using its built-in Atom feed reader.

API Security

At the moment our StarbucksPaymentAPI is exposed over HTTP without any form of authentication. But that’s not how real world online payment systems are designed. We should restrict users to access this API over HTTPS and ideally we should introduce some form of authentication into the equation too. Restricting the payment API to HTTPS is easy. Simply add the protocol=”https” attribute to the resource definition, and immediately the resource is accessible only over HTTPS.

<resource uri-template="/order/{orderId}" methods="GET PUT"
                faultSequence="StarbucksFault" protocol="https">

Now try accessing the payment API over HTTP as follows.

curl -v -X PUT-d @payment.xml -H "Content-type: application/xml" https://localhost:8281/payment/order/order-id

The message will not even get dispatched to the resource. It will be forwarded to the main sequence of the ESB which is configured to return a 403 Forbidden response.

HTTP/1.1 403 Forbidden
Content-Type: application/xml; charset=UTF-8
Host: 127.0.0.1:8281
Date: Wed, 15 Aug 2012 14:18:28 GMT
Server: Synapse-HttpComponents-NIO
Transfer-Encoding: chunked
Connection: Keep-Alive

<?xml version="1.0" encoding="UTF-8"?>
<error xmlns="http://ws.apache.org/ns/synapse">Invalid request</error>

The only way you can now access the StarbucksPaymentAPI is via HTTPS. Invoke the following Curl command to try this out. You should get the usual 201 response.

curl -v  -X PUT -d @payment.xml -H "Content-type: application/xml" -k -X PUT https://localhost:8244/payment/order/order-id

Now that we have some transport level security implemented for our payment API, let’s see how we can add some authentication logic into the solution. REST APIs in WSO2 ESB support a concept known as handlers. One or more handlers can be engaged on a given API where they can intercept the message flows and add various QoS functionality into the APIs. For the purpose of this article, we are going to use a custom handler that provides HTTP basic authentication functionality.

Shutdown the ESB server if it’s already running. Download the WSO2-REST-BasicAuth-Handler-1.0-SNAPSHOT.jar and copy it into the repository/components/lib directory of the ESB installation. Now add the following handler definition to the StarbucksPaymentAPI.

<handlers>
    <handler class="org.wso2.rest.BasicAuthHandler"/>
</handlers>

When you restart the ESB, the StarbucksPaymentAPI will be secured with HTTP basic authentication. Try accessing it as follows, which will result in a 401 Unauthorized response.

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Basic realm="WSO2 ESB"
Date: Wed, 15 Aug 2012 14:35:25 GMT
Server: Synapse-HttpComponents-NIO
Transfer-Encoding: chunked

You must pass in the basic auth credentials (user: admin, password: admin) for the API to be accessible.

curl -v -k -H "Authorization: Basic YWRtaW46YWRtaW4=" https://localhost:8244/payment/order/order-id

You can try this out using a web browser as well. Note how the browser will display the standard authentication dialog when you try to access it without passing in the credentials.

Since we have the basic auth handler engaged on our payment API, all the requests to the API are first processed through the handler. This handler checks whether the required basic auth headers are present on the message. If not it returns a 401 response and terminates the flow. You can find the source code of this custom handler here.

The above custom handler is not a very sophisticated one. You can see in the source code that the user credentials are actually hard coded into it. In a real world situation, the security handlers should contact some kind of a database or an identity provider to authenticate and authorize users. For WSO2 API Manager product, we have developed such an advanced API handler. That implementation secures APIs using OAuth and works in conjunction with the WSO2 Identity Server components.

The API handler concept can be used to implement various other QoS features too. Important non-functional requirements such as throttling, monitoring and usage tracking can be implemented as separate handlers and engaged on your APIs.

Caching

The response time of some of the API calls can be significantly improved by introducing some caching features into our solution. Webber’s article suggests that the list of pending orders should be cached so that the Atom feed generation can be performed without overloading the backend servers. We can implement this in WSO2 ESB using the cache mediator. The cache mediator stores the responses in an in-memory cache keyed by the DOM hash of the requests. Therefore if the same request is sent multiple times, only the first invocation will be mediated to the backend service. All subsequent calls will be served from the cache. Following example shows how the cache mediator can be engaged in one of the ESB sequences.

<cache timeout="20" scope="per-host" collector="false" hashGenerator="org.wso2.caching.digest.DOMHASHGenerator">
    <implementation type="memory" maxSize="100"/>
</cache>

The number of responses to cache and the cache timeout duration are configurable.

GUI Clients

Now that the application integration is complete, we can start working on some UI clients to consume our REST APIs. From a real world perspective, the UI client could be a website, a standalone desktop application or even a mobile app.

First we need to find a HTTP client library which can be embedded in our client application. The choice of HTTP client library greatly depends on the programming language and platform we are going to use for the UI client. If you decide to go ahead with Java as your development platform, then something like Apache HTTP Client would be your ideal choice. It’s easy to use, well documented and quite stable.

The starbucks-rest-sample.jar is a Java UI client we have developed for the Starbucks OMS. To try it out, download the jar file to your local file system. Then using a shell (Unix/Linux) or a command prompt (Windows) execute the following command:

java -jar starbucks-rest-sample.jar

First you will be asked to select one of “Customer Mode” or “Barista Mode”. Once you have made your choice it should be fairly straight forward to find your way around the application. In the client mode you can place orders, review them, update them and make payments. In the barista mode you can review the list of pending orders, process them and deliver them. You will be able to trace all the wire level HTTP messages on the UI itself.

Feel free to go through the source code of the client application as well. Most of the code related to actual HTTP invocations are in the HTTPUtils class.

Note: This particular GUI client expects the payment API to be exposed over HTTP without any form of authentication. So if you have added any security features to the payment API, please revert those modifications before you attempt to run the client application.

Summary and Conclusion

We started our discussion by looking at how WSO2 middleware supports developing RESTful integrations. We analyzed the REST support available in WSO2 Application Server and WSO2 ESB in detail. We looked at the new REST API concept of WSO2 ESB and how it makes developing RESTful integrations easier. Based on this groundwork we implemented a complete order management solution using WSO2 middleware. While implementing this solution we learnt many important things such as proper use of HTTP methods and response codes, HATEOAS, error handling and REST to SOAP conversion in WSO2 ESB.

Once we got our solution up and running, we started exploring the ways to further improve it. We discussed several concepts such as content negotiation, security and caching in detail and how those features can be incorporated into our WSO2 ESB based solution. Finally we looked at a sample UI application which can be used to interact with the REST APIs we have developed.

From this exercise it becomes clear that WSO2 ESB and rest of the WSO2 platform in general have everything required to build comprehensive solutions based on REST application architecture. WSO2 AS can be used to host native REST applications and WSO2 ESB can be used as a centralized router and mediator of REST calls. Further it can be used as a bridge between RESTful applications and non-RESTful applications. It provides a convenient and powerful way to expose existing systems over clean and well-defined RESTful APIs.

Sample Artifacts

  • Starbucks OMS (web service) - https://svn.wso2.org/repos/wso2/people/hiranya/rest-sample/bin/StarbucksOutletService.aar
  • ESB configuration - https://svn.wso2.org/repos/wso2/people/hiranya/rest-sample/bin/synapse-configs.zip
  • Basic auth handler for REST APIs - https://svn.wso2.org/repos/wso2/people/hiranya/rest-sample/bin/WSO2-REST-BasicAuth-Handler-1.0-SNAPSHOT.jar
  • Starbucks UI client - https://svn.wso2.org/repos/wso2/people/hiranya/rest-sample/bin/starbucks-rest-sample.jar

Author

Hiranya Jayathilaka, Project Lead, WSO2 Enterprise Service Bus

Disclaimer

Starbucks and the Starbucks logo (used in the UI client) are registered trademarks of Starbucks

 

About Author

  • Hiranya Jayathilaka
  • PhD student
  • Department of Computer Science at UC Santa Barbara