Platform Test Automation using WSO2 Test Automation Framework

  • By Nuwan Wimalasekara
  • 6 Mar, 2014

Table of contents

  • Introduction
  • WSO2 Test Automation Framework
  • Test Scenario
  • Prerequisites
  • Test Project structure
  • pom.xml configurations
  • automation.xml configuration
  • Writing Test class
  • Executing test class using testng.xml


When it comes to continuous integration and continuous delivery of high quality software, Test Automation has became one of the most important aspects in the testing phase of the software development life cycle. Automation testing will speedup the development cycle by ensuring existing functionalities of the system under tests are working fine, and deliver faster feedback.

The WSO2 Test Automation Framework is a test framework based on TestNG for implementing an automated integration test on the carbon platform involving more than one product. You just need to build the required steps of the test scenario and you can implement test cases for any WSO2 server features.

WSO2 Test Automation Framework

The WSO2 Test Automation Framework consists of four major components as explained below and can be used as a dependency in your maven test project. The test framework will satisfy preconditions that need to be met for the test cases, such as server startup, users and tenant population, setting up keystores, etc. and provides a lot of utility classes to bring down test case complexity.

  1. org.wso2.carbon.automation.test.api - Provide Admin Service Client classes to invoke WSO2 server features as a web service. Simulation of any functionality that is provided through the management console is possible via Admin Services APIs
  2. org.wso2.carbon.automation.engine - Provide the core features of the test framework
  3. org.wso2.carbon.automation.extensions - Provide various extensions required for testing like wso2 server management, axis2 server instance, ActiveMQ Server instance, jms clients, etc.
  4. org.wso2.carbon.automation.test.utils - Provides the utility classes for test cases

Test scenario

Using the test framework, we can test from a simple test scenario like integrating one product to a complex scenario like integrating more than one product as a platform. You can write test classes for most test scenarios using the WSO2 test framework.

The WSO2 platform provide great support on web services. Most web services are fronted by a proxy service. Here we are going to implement a test class for the scenario explained below.

The requirement is to deploy a webservice(SimpleStockQuoteService) and invoke the service on AS through the proxy service(stockQuote) deploying on ESB. The response must be verified once the service is invoked by a service client.

The above is a test scenario involving multiple products. We assume both the WSO2 ESB and WSO2 AS servers are up and running at the time of test execution. If you want to automate the server startup as well, you can write a pluggable extension for this purpose.


Test Automation Framework 4.3.0 or onwards

WSO2 ESB 4.8.1 or onwards

WSO2 AS 5.2.1 or onwards

Test project structure

First, we need to create a maven project to implement the test classes. Surefire plugin is used for test execution.

Project directory structure

Test configuration

There are a few resource files and directories you would need to add under the resources directory. In order to manage the test artifacts, you would need to maintain directories with a given naming convention. For the above scenario, AS and ESB directories have been created under the artifacts directory. Thereafter, the artifacts used for the test scenario can be managed from those directories. SimpleStockQuoteSerice.aar is copied to the AS directory and synapse.xml is copied to the ESB directory.

Since https transport is used for admin service, we need to keep the keystores file under the resources directory. Thus, client-truststore.jks and wso2carbon.jks files need to be copied to resources/keystores/products directory. Those files can be located at $CARBON_HOME/repository/resources/security.

Then keep the automation.xml and automation_mapping.xsd under the resources directory.

pom.xml configurations

The path of the resource directory of the test module is set as a Java system property.
The property framework.resource.location must be set in surefire configuration as shown below to access the file system location of the resource directory within the framework and test classes that contain the artifacts.

pom.xml configuration


4.0.0org.demoorg.demo.tests.sample1.0.0Sample Test Modulejarmaven-surefire-plugin2.12.4false-Xms512m -Xmx1024m -XX:MaxPermSize=128mtruefalsefalsesrc/test/resources/testng.xmlframework.resource.location

automation.xml configuration

automation.xml is the file where the test framework get the test configuration values. ex: server information, user or tenant information, environment details, etc. So we need to configure automation.xml file properly according to your test platform.

executionEnvironment should be set as platform since this test case required more than one WSO2 servers, and those servers are not started by the test framework itself at the test run time. Therefore, ESB and AS server information must be set according to your servers under the platform section, e.g ip, ports.

Correct user information must be set as your platform under the userManagement section. Note that you need to set the correct username and password for super tenant, then other users are populated at the test runtime by the test framework if they do not exist.


Writing test class

Now we have created the maven project for test suite.

We need to copy the SimpleStockQuoteSerice.aar and synapse.xml file accordingly - artifacts/AS and artifacts/ESB. Therefore, we can use those artifacts within the test class.

We can add a Java class with the below TestNG annotation to make it a test class.

@BeforeClass - Method will execute before all the Test method

@AfterClass - Method will execute after all the Test method

@Test - test method and should contains the test scenario

Here we initialize the ESB and AS environment as super user(admin) under BeforeClass and initializing the admin service client classes required for test scenario.

AARServiceUploaderClient - to upload aar file into AS.
ServiceAdminClient - to manage deployed services like list services, delete services, etc.

ESBTestCaseUtils - is the utility class provided for synapse artifact deployment in ESB integration test. It will upload the given synapse artifact and wait until the deployment is done as well.



There are three test methods in this class.

uploadWebService() - To test the webservice deployment on AS

uploadSynapseConfig() - To test the synapse artifact deployment on ESB

invokeService() - To test the service invocation. This method has used a attribute called dependsOnMethods in Test annotation to run the test method only if uploadWebService and uploadSynapseConfig is executed successfully.

public class SampleTestCase {
   private final String SERVICE_NAME = "SimpleStockQuoteService";
   private AutomationContext esbContext;
   private AutomationContext asContext;

   private ESBTestCaseUtils esbTestUtil;

   private String esbSession;
   private String asSession;

   private AARServiceUploaderClient aarServiceUploaderClient;
   private ServiceAdminClient serviceAdmin;

   public void init() throws Exception {
       //initializing the ESB and AS environments from the super admin
       esbContext = new AutomationContext("ESB", TestUserMode.SUPER_TENANT_ADMIN);
       asContext = new AutomationContext("AS", TestUserMode.SUPER_TENANT_ADMIN);

       //getting session cookies
       esbSession = esbContext.login();
       asSession = asContext.login();

       esbTestUtil = new ESBTestCaseUtils();
       aarServiceUploaderClient = new AARServiceUploaderClient(asContext.getContextUrls().getBackEndUrl()
               , asSession);
       serviceAdmin = new ServiceAdminClient(asContext.getContextUrls().getBackEndUrl()
               , asSession);


   public void uploadWebService() throws Exception {
       //deploying axis2 service on AS
       aarServiceUploaderClient.uploadAARFile(SERVICE_NAME + ".aar"
               , TestConfigurationProvider.getResourceLocation("AS") + File.separator + SERVICE_NAME + ".aar", "");
               , asSession, SERVICE_NAME), "Service Deployment failed");

   public void uploadSynapseConfig() throws Exception {
       //deploying synapse configuration on ESB
               , esbContext.getContextUrls().getBackEndUrl(), esbSession);


   @Test(dependsOnMethods = {"uploadSynapseConfig", "uploadWebService"})
   public void invokeService() throws Exception {
       //invoking the service through the proxy service
       AxisServiceClient client = new AxisServiceClient();

       //building the proxy service url
       String proxyServiceUrl =  esbContext.getContextUrls().getServiceUrl() + "/stockQuote";

       OMElement response = client.sendReceive(getStockQuoteRequest("WSO2")
               , proxyServiceUrl, "urn:getQuote");

       //verifying the response
               getFirstChildWithName(new QName("https://services.samples/xsd", "name")).getText()
               , "WSO2 Company", "Response mismatched");

   public void cleanup() throws Exception {
       //deleting the axis2 service and proxy service
       serviceAdmin.deleteService(new String[]{SERVICE_NAME});
       esbTestUtil.deleteProxyService(esbContext.getContextUrls().getBackEndUrl(), esbSession, "stockQuote");

       //getting the service deployment delay configured in automation.xml
       int serviceDeploymentUndeploymentDelay = TestConfigurationProvider.getServiceDeploymentDelay();

       //verifying whether axis2 and proxy service undeployed within the configured time period
       Assert.assertTrue(ServiceDeploymentUtil.isServiceUnDeployed(asContext.getContextUrls().getBackEndUrl(), asSession
               , SERVICE_NAME, serviceDeploymentUndeploymentDelay), "Service Undeployment failed");

               , esbSession, "stockQuote"), "Proxy undeployment failed");

   private OMElement getStockQuoteRequest(String symbol) {
       OMFactory fac = OMAbstractFactory.getOMFactory();
       OMNamespace omNs = fac.createOMNamespace("https://services.samples", "ns");
       OMElement method = fac.createOMElement("getQuote", omNs);
       OMElement value1 = fac.createOMElement("request", omNs);
       OMElement value2 = fac.createOMElement("symbol", omNs);

       value2.addChild(fac.createOMText(value1, symbol));

       return method;

Executing test class using testng.xml

We have completed the test class implementation. Now you need to run the test class to verify server functionality. Surefire plugin is used for executing the test class and an xml file called testng.xml is used to manage the test classes to be executed. In testng.xml, we can add any number of test classes and it is considered a test suite.

In the below testng.xml configuration, org.demo.SampleTestCase is defined under test element. Likewise, you can add any number of test classes or any number of test elements. Then it will execute the test classes in the test element as the order.

There is a special configuration under the listener’s section. The defined listeners are provided by the test framework and those are suppose to provide the specific task at the specific phases at the test suite execution. Therefore, it is better to keep those configuration as is.

Then go to the test maven project and issuing mvn clean install will execute the test classes defined in testng.xml and after test execution you can browse the surefire reports at target/surefire-reports.

Note: Please make sure that the ESB and AS servers are up and running according to the configuration in automation.xml. In this sample, the ESB should be started with 0 offset and the AS should be started with the offset value 2.


Now you have completed the test class; you can make sure the scenario is working by executing the tests. Also test suite can be updated by adding more test classes.

Refer to this sample code for the above test case.

About Author

  • Nuwan Wimalasekara
  • Software Engineer
  • WSO2