2015/05/13
13 May, 2015

[Tutorial] Hosting RESTful Web Services Using OAuth with the WSO2 Platform

  • Suhan Dharmasuriya
  • Associate Technical Lead - WSO2

In this use case a RESTful web service is hosted using OAuth authentication. A request will be saved into a database and the response will be returned to the service consumer. This request and response should be in JSON format.

Introduction

In this use case a RESTful web service is hosted using OAuth authentication. A request will be saved into a database and the response will be returned to the service consumer. This request and response should be in JSON format.

Figure 01

In this scenario three WSO2 products are used, WSO2 Enterprise Service Bus (ESB) that will host the RESTful web service, WSO2 Identity Server (IS) that will manage the OAuth authentication and WSO2 Data Services Server (DSS) that will manage data services. A locally installed MySQL database will be used throughout the scenario and SOAP UI is used to send JSON requests to the REST API. The complete scenario will be tested on a single computer instance.

Running multiple WSO2 products on the same computer instance

Port offsets used for the servers as follows;

Server Version Offset URL
WSO2 Enterprise Service Bus 4.8.1 0 https://localhost:9443/carbon
WSO2 Identity Server 5.0.0 1 https://localhost:9444/carbon
WSO2 Data Services Server 3.2.1 10 https://localhost:9453/carbon

You can simply change the port offset through <PRODUCT_HOME>/repository/conf/carbon.xml by changing the <Ports> section, <Offset>1&lt/Offset>. Once changed start/restart the server.

These port offset numbers can be changed according to your choice. If you set your port offset of any WSO2 product to 2, your admin console URL will be https://localhost:9445/carbon, i.e. port -> 9443+<offset> = 9445

Preparing the MySQL database

  1. Install MySQL database if it isn’t installed on your computer.
  2. Log in to MySQL using the following command.

    > mysql -u root -p

    Enter password: <rootPassword>

  3. Issue the following commands to create the database and relevant tables.

    create database dss_sample;

    use dss_sample;

    CREATE TABLE Employee(EmployeeID int PRIMARY KEY AUTO_INCREMENT, FirstName varchar(255), LastName varchar(255),Team varchar(255));

Configuring WSO2 DSS

  1. Download WSO2 DSS 3.2.1 from here. If you already have the product zip file (wso2dss-3.2.1.zip) continue with the next step.
  2. Unzip the product to a path containing no spaces in the path name. This is your <DSS_HOME>
  3. Set port offset to 10 as instructed above.
  4. Download the MySQL connector jar here and copy it to <DSS_HOME>/repository/components/lib/
  5. Start the WSO2 DSS server.

    To start the server, your have to run the script wso2server.bat (on Windows) or wso2server.sh (on Linux/Solaris) from the <DSS_HOME>/bin folder.

  6. Log in to DSS by using the default credentials (username: admin/ password: admin).
  7. Creating the data source

  8. Create a datasource as follows by referring to the MySQL database.

    Configure -> Data Sources -> Add Data Source

    Data Source Type RDBMS
    Name EmployeeDataSource
    Data Source Provider default
    Driver com.mysql.jdbc.Driver
    URL jdbc:mysql://127.0.0.1:3306/dss_sample
    Username root
    Password rootPassword


    Figure 02

  9. Generating the data service

  10. Auto generate the Data service from the datasource created above.

    Main -> Services -> Data Services -> Generate

    Carbon datasource details

    Attribute Value
    Carbon Datasource(s) EmployeeDataSource
    Database Name dss_sample


    Figure 03

    Select table(s)

    Click 'Next'.

    Figure 04

    Select service generation mode

    Select -> Single Service

    Attribute Value
    Data Service Namespace https://employees.us.wso2.com
    Data Service Name EmployeesDataService


    Figure 05

  11. Click ‘Next’.

    You will see a ‘Service(s) Deployed Successfully’ message.

  12. Click ‘Finish’.

Now the DSS configuration is complete.

Once you click on ‘Finish’, you will be redirected to the deployed services dashboard.

Figure 06

Now click on EmployeesDataService from the list shown and go to its service dashboard.

Figure 07

Under operations you can see that the five default operations listed below are automatically created when the data service was generated from the given data source given. We can use them for our use case without having to define new queries and operations.

  • insert_Employee_operation
  • update_Employee_operation
  • select_with_key_Employee_operation
  • delete_Employee_operation
  • select_all_Employee_operation

Figure 08

XML configuration of the data source can be viewed by clicking on ‘Edit Data Service (XML Edit)’ link. Note that this configuration is auto generated by DSS and shown below for reference only.

  
<data name="EmployeesDataService" serviceNamespace="https://employees.us.wso2.com">
   <config id="default">
      <property name="carbon_datasource_name">EmployeeDataSource</property>
   </config>
   <query id="select_all_Employee_query" useConfig="default">
      <sql>SELECT EmployeeID, FirstName, LastName, Team FROM Employee</sql>
      <result element="EmployeeCollection" rowName="Employee">
         <element column="EmployeeID" name="EmployeeID" xsdType="xs:integer"/>
         <element column="FirstName" name="FirstName" xsdType="xs:string"/>
         <element column="LastName" name="LastName" xsdType="xs:string"/>
         <element column="Team" name="Team" xsdType="xs:string"/>
      </result>
   </query>
   <query id="update_Employee_query" useConfig="default">
      <sql>UPDATE Employee SET FirstName=?,LastName=?,Team=? WHERE EmployeeID=?</sql>
      <param name="FirstName" ordinal="1" sqlType="STRING"/>
      <param name="LastName" ordinal="2" sqlType="STRING"/>
      <param name="Team" ordinal="3" sqlType="STRING"/>
      <param name="EmployeeID" ordinal="4" sqlType="INTEGER"/>
   </query>
   <query id="select_with_key_Employee_query" useConfig="default">
      <sql>SELECT EmployeeID, FirstName, LastName, Team FROM Employee WHERE EmployeeID=?</sql>
      <result element="EmployeeCollection" rowName="Employee">
         <element column="EmployeeID" name="EmployeeID" xsdType="xs:integer"/>
         <element column="FirstName" name="FirstName" xsdType="xs:string"/>
         <element column="LastName" name="LastName" xsdType="xs:string"/>
         <element column="Team" name="Team" xsdType="xs:string"/>
      </result>
      <param name="EmployeeID" ordinal="1" sqlType="INTEGER"/>
   </query>
   <query id="delete_Employee_query" useConfig="default">
      <sql>DELETE FROM Employee WHERE EmployeeID=?</sql>
      <param name="EmployeeID" ordinal="1" sqlType="INTEGER"/>
   </query>
   <query id="insert_Employee_query" returnGeneratedKeys="true" useConfig="default">
      <sql>INSERT INTO Employee(FirstName,LastName,Team) VALUES(?,?,?)</sql>
      <result element="GeneratedKeys" rowName="Entry" useColumnNumbers="true">
         <element column="1" name="ID" xsdType="integer"/>
      </result>
      <param name="FirstName" ordinal="1" sqlType="STRING"/>
      <param name="LastName" ordinal="2" sqlType="STRING"/>
      <param name="Team" ordinal="3" sqlType="STRING"/>
   </query>
   <operation name="update_Employee_operation">
      <call-query href="update_Employee_query">
         <with-param name="EmployeeID" query-param="EmployeeID"/>
         <with-param name="FirstName" query-param="FirstName"/>
         <with-param name="Team" query-param="Team"/>
         <with-param name="LastName" query-param="LastName"/>
      </call-query>
   </operation>
   <operation name="select_all_Employee_operation">
      <call-query href="select_all_Employee_query"/>
   </operation>
   <operation name="insert_Employee_operation">
      <call-query href="insert_Employee_query">
         <with-param name="FirstName" query-param="FirstName"/>
         <with-param name="Team" query-param="Team"/>
         <with-param name="LastName" query-param="LastName"/>
      </call-query>
   </operation>
   <operation name="delete_Employee_operation">
      <call-query href="delete_Employee_query">
         <with-param name="EmployeeID" query-param="EmployeeID"/>
      </call-query>
   </operation>
   <operation name="select_with_key_Employee_operation">
      <call-query href="select_with_key_Employee_query">
         <with-param name="EmployeeID" query-param="EmployeeID"/>
      </call-query>
   </operation>
</data>

Configuring WSO2 IS

  1. You can download WSO2 IS 5.0.0 from here. If you already have the product zip file (wso2is-5.0.0.zip) continue with the next step.
  2. Unzip the product to a path containing no spaces in the path name. This is your <IS_HOME>
  3. Set port offset to 1 as instructed above.
  4. Start the WSO2 IS server.

    To start the server you have to run the script wso2server.bat (on Windows) or wso2server.sh (on Linux/Solaris) from the <IS_HOME>/bin folder.

  5. Log in to IS using the default credentials (username: admin/ password: admin).
  6. Adding a service provider

  7. Add a service provider

    Main -> Identity -> Service Providers -> Add

    Figure 09

  8. Once you add necessary information, register the service provider in IS.

    Once you click on ‘Register’ you will be redirected to a service provider configuration page (Service Providers).

    Figure 10

  9. Next go to Inbound Authentication Configuration -> OAuth/OpenID Connect Configuration and click ‘Configure’.
  10. Register the new application

    Now you will be redirected to the ‘Register New Application’ page.

    Figure 11

  11. Fill the details as shown above and click ‘Add’ to register your application.
  12. Once you register your application process you can view the OAuth client key and client secret. If you cannot see the client secret click on the ‘Show’ button to view the text.

    Figure 12

    E.g.:

    OAuth Client Key CL7hb_C7hAmNryv7dzVm9VDT6xMa
    OAuth Client Secret Ikb8fAFPKPSaapdznNhBGTExJnMa

    We need the above two parameter values to generate an access token.

    Generating the access token

  1. Issue the following cURL command
    curl -v -k -X POST --user CL7hb_C7hAmNryv7dzVm9VDT6xMa:Ikb8fAFPKPSaapdznNhBGTExJnMa -H "Content-Type: application/x-www-form-urlencoded;charset=UTF-8" -d 'grant_type=password&username=admin&password=admin' https://localhost:9444/oauth2/token
    * Hostname was NOT found in DNS cache
    *   Trying 127.0.0.1...
    * Connected to localhost (127.0.0.1) port 9444 (#0)
    * TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
    * Server certificate: localhost
    * Server auth using Basic with user 'CL7hb_C7hAmNryv7dzVm9VDT6xMa'
    > POST /oauth2/token HTTP/1.1
    > Authorization: Basic Q0w3aGJfQzdoQW1Ocnl2N2R6Vm05VkRUNnhNYTpJa2I4ZkFGUEtQU2FhcGR6bk5oQkdURXhKbk1h
    > User-Agent: curl/7.37.1
    > Host: localhost:9444
    > Accept: */*
    > Content-Type: application/x-www-form-urlencoded;charset=UTF-8
    > Content-Length: 49
    > 
    * upload completely sent off: 49 out of 49 bytes
    
  2. Extract the access_token part i.e. 46bad0faaf58085936b22d0dbb953fa
  3. This is to be included in the authorization header value field when you are calling the REST API in the following steps.

    You have just completed the WSO2 IS configuration.

    Note that the expires_in value is 3599999699 because we have extended the token expiration parameter in WSO2 IS. Default value is 3600 seconds.
    This is an optional configuration. You can skip this step.

    Extending application token expiration period

    To extend the token expiration parameter in WSO2 IS follow the steps below;

  1. Open <IS_HOMEgt;/repository/conf/identity.xml to edit it.
  2. Under <OAuth> find the <AccessTokenDefaultValidityPeriod> and <UserAccessTokenDefaultValidityPeriod> tags1.
  3. Set the values to a larger number. Note that the default value is 3600 seconds.
  4. E.g.:

    <!-- Default validity period for user access tokens in seconds -->
    <AccessTokenDefaultValidityPeriod>3600000000</AccessTokenDefaultValidityPeriod>
    <!-- Default validity period for application access tokens in seconds -->
    <UserAccessTokenDefaultValidityPeriod>3600000000</UserAccessTokenDefaultValidityPeriod>
    

Configuring WSO2 ESB

  1. You can download WSO2 ESB 4.8.1 from here. If you already have the product zip file (wso2esb-4.8.1.zip) continue with the next step.
  2. Unzip the product to a path containing no spaces in the path name. This is your <ESB_HOME>
  3. Start the WSO2 ESB server.

    To start the server, your have to run the script wso2server.bat (on Windows) or
    wso2server.sh (on Linux/Solaris) from the <ESB_HOME>/bin folder.

  4. Log in to the ESB using default credentials (username: admin/ password: admin).
  5. Creating Sequences

  6. Create the two sequences below and save them in the ESB Registry. Refer to [2] on how to save a sequence in the registry.

    Sequence Name: ProcessPayloadForEmpDSSSequence

    <sequence xmlns="http://ws.apache.org/ns/synapse">
       <oauthService remoteServiceUrl="https://localhost:9444/services" username="admin" password="admin"></oauthService>
       <payloadFactory media-type="xml">
          <format>
             <p:insert_Employee_operation xmlns:p="https://employees.us.wso2.com">            
                <xs:FirstName xmlns:xs="https://employees.us.wso2.com">$1</xs:FirstName>         
                <xs:LastName xmlns:xs="https://employees.us.wso2.com">$2</xs:LastName>           
                <xs:Team xmlns:xs="https://employees.us.wso2.com">$3</xs:Team>         
             </p:insert_Employee_operation>
          </format>
          <args>
             <arg expression="$.employee.firstName" evaluator="json"></arg>
             <arg expression="$.employee.lastName" evaluator="json"></arg>
             <arg expression="$.employee.team" evaluator="json"></arg>
          </args>
       </payloadFactory>
    </sequence>
    

    Sequence Name: ProcessResponseFromEmpDSSService

    <sequence xmlns="http://ws.apache.org/ns/synapse">
       <log></log>
       <payloadFactory media-type="json">
          <format>                {"EmployeeRecord":{"EmployeeID":$1, "Status":"Successfully created"}}            </format>
          <args>
             <arg expression="$.GeneratedKeys.Entry.ID" evaluator="json"></arg>
          </args>
       </payloadFactory>
       <property name="messageType" value="application/json" scope="axis2"></property>
    </sequence>
    

    Figure 13

  7. Adding an API

  8. Add an API from Main -> Service Bus -> APIs -> Add API -> Switch to Source View API Configuration. This is shown below (as your RESTful service);
    <api name="OrganizationalInfoAPI" context="/internal">
          <resource methods="POST" uri-template="/employees">
             <inSequence>
                <sequence key="conf:/ProcessPayloadForEmpDSSSequence"/>
                <call>
                   <endpoint>
                      <address uri="https://10.100.5.175:9773/services/EmployeesDataService"
                               format="soap12"/>
                   </endpoint>
                </call>
                <sequence key="conf:/ProcessResponseFromEmpDSSService"/>
                <respond/>
             </inSequence>
          </resource>
    </api>
    

    Figure 14

Sending a JSON request to the REST API

There are two ways to send the requests to the REST API;

  • Through SOAP UI
  • By issuing a cURL command

Through SOAP UI

  1. Create a new REST project.

    Figure 15

  2. Enter https://10.100.5.175:8280/internal/employees as the URI.
  3. Select the method POST.
  4. Select application/json as the media type and add the following payload in the text field;

    {"employee":{"firstName":"Jackie","lastName":"Chan","team":"Finance"}}

    Figure 16

  5. Add a header from the headers tab.
    Header: Authorization
    Value: Bearer 46bad0faaf58085936b22d0dbb953fa

Note that the 46bad0faaf58085936b22d0dbb953fa string is the access token we generated using the cURL command sent to WSO2 IS.

Once you send the request you will get our customized response message similar to the one below. Make sure to view the RAW response from SOAP UI.

{"EmployeeRecord": {
   "EmployeeID": 39,
   "Status": "Successfully created"
}}

Figure 17

This is the Raw response received after attempting another insert operation.

Figure 18

Issuing a cURL command

Instead of using the SOAP UI, you can issue the following cURL command;

curl -v -X POST -H "Authorization: Bearer 46bad0faaf58085936b22d0dbb953fa" -H "Content-Type:application/json" -d '{"employee":{"firstName":"Suhan","lastName":"Dharmasuriya","team":"TestAuto"}}' https://10.100.5.175:8280/internal/employees

* Hostname was NOT found in DNS cache
*   Trying 10.100.5.175...
* Connected to 10.100.5.175 (10.100.5.175) port 8280 (#0)
> POST /internal/employees HTTP/1.1
> User-Agent: curl/7.37.1
> Host: 10.100.5.175:8280
> Accept: */*
> Authorization: Bearer 46bad0faaf58085936b22d0dbb953fa
> Content-Type:application/json
> Content-Length: 78
> 
* upload completely sent off: 78 out of 78 bytes

Follow-up

Query the MySQL Employee table data

Figure 19

The wire log trace in ESB console is shown below.
To see how to enable wire logs, refer to [3].

[2015-04-29 16:47:56,767] DEBUG - wire >> "POST /internal/employees HTTP/1.1[\r][\n]"
[2015-04-29 16:47:56,768] DEBUG - wire >> "Accept-Encoding: gzip,deflate[\r][\n]"
[2015-04-29 16:47:56,768] DEBUG - wire >> "Content-Type: application/json[\r][\n]"
[2015-04-29 16:47:56,768] DEBUG - wire >> "Authorization: Bearer 46bad0faaf58085936b22d0dbb953fa[\r][\n]"
[2015-04-29 16:47:56,768] DEBUG - wire >> "Content-Length: 71[\r][\n]"
[2015-04-29 16:47:56,768] DEBUG - wire >> "Host: 10.100.5.175:8280[\r][\n]"
[2015-04-29 16:47:56,768] DEBUG - wire >> "Connection: Keep-Alive[\r][\n]"
[2015-04-29 16:47:56,768] DEBUG - wire >> "User-Agent: Apache-HttpClient/4.1.1 (java 1.5)[\r][\n]"
[2015-04-29 16:47:56,768] DEBUG - wire >> "[\r][\n]"
[2015-04-29 16:47:56,769] DEBUG - wire >> "{"employee":{"firstName":"Jackie","lastName":"Chan","team":"Finance"}}[\n]"
[2015-04-29 16:47:56,963] DEBUG - wire JackieChanFinance[\r][\n]"
[2015-04-29 16:47:56,964] DEBUG - wire > "HTTP/1.1 200 OK[\r][\n]"
[2015-04-29 16:47:56,986] DEBUG - wire >> "Content-Type: application/soap+xml;charset=UTF-8[\r][\n]"
[2015-04-29 16:47:56,987] DEBUG - wire >> "Transfer-Encoding: chunked[\r][\n]"
[2015-04-29 16:47:56,987] DEBUG - wire >> "Date: Wed, 29 Apr 2015 11:17:56 GMT[\r][\n]"
[2015-04-29 16:47:56,987] DEBUG - wire >> "Server: WSO2 Carbon Server[\r][\n]"
[2015-04-29 16:47:56,987] DEBUG - wire >> "[\r][\n]"
[2015-04-29 16:47:56,988] DEBUG - wire >> "dc[\r][\n]"
[2015-04-29 16:47:56,988] DEBUG - wire >> "40[\r][\n]"
[2015-04-29 16:47:56,988] DEBUG - wire >> "22[\r][\n]"
[2015-04-29 16:47:56,988] DEBUG - wire >> "[\r][\n]"
[2015-04-29 16:47:56,990] DEBUG - wire >> "0[\r][\n]"
[2015-04-29 16:47:56,990] DEBUG - wire >> "[\r][\n]"
[2015-04-29 16:47:56,993]  INFO - LogMediator To: https://www.w3.org/2005/08/addressing/anonymous, WSAction: , SOAPAction: , MessageID: urn:uuid:29e5d85b-3bdb-4468-a1d5-043ed3aa3aa7, Direction: request
[2015-04-29 16:47:56,996]  INFO - LogMediator To: https://www.w3.org/2005/08/addressing/anonymous, WSAction: , SOAPAction: , MessageID: urn:uuid:29e5d85b-3bdb-4468-a1d5-043ed3aa3aa7, Direction: request
[2015-04-29 16:47:57,002] DEBUG - wire 

Overview of the products used

WSO2 ESB

WSO2 Enterprise Service Bus (WSO2 ESB) is the main integration backbone of the WSO2 platform. Unlike some Enterprise Service Buses in the market, WSO2 ESB supports all enterprise integration patterns which is one of its main competitive advantages. System administrators and SOA architects can simply and easily configure message routing, virtualization, intermediation, transformation, logging, task scheduling, load balancing, fail-over routing, event brokering, and many more with WSO2 ESB.

What if you want to go even further? Is it possible with WSO2 ESB? Yes, there are extension points to WSO2 ESB such as, script mediator, class/custom mediators, connectors, custom tasks, message builders/formatters and it even allows custom transports. Most importantly WSO2 ESB can co-exist with legacy systems and it allows to fully harness their current backend capabilities.

WSO2 DSS

WSO2 Data Services Server (WSO2 DSS) is capable of dynamically wrapping different data stores with a service layer, integrating data sources (RDBMS or NoSQL), creating composite data views, and hosting data services. WSO2 DSS will bear the weight and complexities of handling internal and external consumers of the data stores.

WSO2 IS

WSO2 Identity Server (WSO2 IS) latest state of the art security and identity management of enterprise web applications, services and APIs. Identity Server acts as an Enterprise Identity Bus (EIB) a central backbone to connect and manage multiple identities regardless of the standards on which they are based.

Conclusion

The WSO2 integration platform is capable of handling more complex scenarios than the one explained in this tutorial.

By integrating WSO2 product stack into your business it will allow you to harness your full potential, to comply with standards and lift your business to new heights.

References

[1] https://wso2.org/jira/browse/IDENTITY-2798

[2] https://suhan-opensource.blogspot.com/2015/05/wso2-esb-how-to-save-sequence-in.html

[3] https://suhan-opensource.blogspot.com/2015/03/how-to-get-wire-logs-from-wso2-esb.html

 

About Author

  • Suhan Dharmasuriya
  • Associate Technical Lead
  • WSO2 Lanka (Pvt) Ltd.