2014/05/19
19 May, 2014

Automation of Custom Environments with WSO2 Test Automation Framework

  • Malintha Adikari
  • Software Engineer - WSO2
Archived Content
This article is provided for historical perspective only, and may not reflect current conditions. Please refer to relevant product page for more up-to-date product information and resources.

Applies to

WSO2 Test Automation Framework 4.3.x

Table of contents

  • Introduction
  • WSO2 Test Automation Framework
  • Extensibility feature of Test Automation Framework
    • Developing pluggable modules
  • Storing and retrieving custom configuration details for tests
    • Adding configurations to automation.xml
    • Retrieving configurations from automation.xml
  • Creating custom test environments based on configuration details
  • Summary

Introduction

The main goal of a software enterprise is to deliver high quality software within a short period of time. Reliable test automation solutions ensure that you will achieve this goal comfortably. Traditional test automation solutions support the development of automated tests, but do not provide facilities to maintain automated tests in different test environments. Adopting the same set of test cases for different test environments is an overhead for test developers with traditional test automation systems.

The comprehensive test automation framework which is developed for supporting both automated test development and maintaining those automated tests for different test environments would fill the gap between traditional test automation systems and today’s enterprise test automation requirements.

WSO2 Test Automation Framework

WSO2 Test Automation Framework performs equally in all stages of the deployment cycles. It seamlessly supports the need for platform wide test execution. The test automation framework provides facilities to automate tests over custom environments by

  1. Providing extensibility feature for users through TestNG listeners
  2. Providing a single location to store all configuration details used in tests and providing APIs to retrieve stored configurations within tests
  3. Facilitating users to create custom test environments based on configuration details retrieved from the configuration store

Extensibility feature of Test Automation Framework

Use case

A user wants to test a WSO2 ESB proxy service. He is using an external Apache Axis2 server as his backend server for this ESB proxy service. He plans to automate his test scenarios using an automation system. In the test automating process, he has to prepare the test environment (backend server) before he start executing tests . He decides to startup the Axis2 server programmatically in the beginning of his test scripts. He has to embed the backend server startup logic inside his test scripts though it is not part of his test scenario. Later, he wants to change his backend server to an Apache CXF server.

Now he has to change every test case where the backend server startup logic had been embedded in order to adapt his test cases to the new environment. He might have to change his test cases if he embedded environment preparation or infrastructure providing logic inside the tests.

With WSO2 Test Automation Framework the user can decouple the test logic from the environment preparation logic. It gives the capability to test writers through the framework's extensibility feature. With this flexibility, he can handover tasks that are not related to the test logic to external modules. Then he can plug those external modules to the test execution flaw by registering them in the configuration storage. They can also be used for multiple test cases hence reducing replications. This capability allows the user to use the same test case for multiple environments with minimal changes.

Developing pluggable modules

The WSO2 Test Automation Framework uses TestNG test listeners to perform tasks in the test execution flow. The test listener manager classes implement default TestNG test listeners. The WSO2 Test Automation Framework provides users with five interfaces for developing pluggable modules. They are as follows;

  • ExecutionListenerExtension
  • ReportListenerExtension
  • SuiteListenerExtension
  • TestListenerExtension
  • TransformListenerExtension

Each interface contains methods to be overridden mapped with default TestNG listeners. Choosing the correct interface to implement depends on the point of the test execution flow where the user needs to plug the module to be developed. For example, if a user wants to plug an external module before test execution starts, it should be implemented as ExecutionListenerExtension. The above mentioned interfaces provided are mapped to TestNGlisteners.

  

public class Axis2ServerExtension implements ExecutionListenerExtension {
             private static final Log log = LogFactory.getLog(Axis2ServerExtension.class);
	
	public void initiate() throws Exception {
              //operation to be initiate the task 
	}

	public void onExecutionStart() throws Exception {
    	 //axis2 server startup logic
	}

	public void onExecutionFinish() throws Exception {
    	 //axis2 server shutting  logic
	}

	private static void handleException(String msg, Exception e) {
    	log.error(msg, e);
    	throw new RuntimeException(msg, e);
	}
}

After developing a module, users should register that module in an automation.xml file in the appropriate listener under the "listenerExtensions" section. For example, let’s assume the newly developed pluggable class's class path is org.wso2.carbon.xxx.Axis2ServerExtension and you want to plug this module at the beginning of the test (before test execution starts). This extension class should be in the classpath and users can add this class as maven dependency in the test project. Then we have to register that module as follows in automation.xml;

  

………………

    	
        	    
            	       org.wso2.carbon..xxx.Axis2ServerExtension 
        	   
    	
    	
        	   
            	   
    	

……………….

Storing and retrieving custom configuration details for tests

WSO2 Test Automation Framework provides a single location to store all configuration details that are to be used inside test cases. Storing configuration details in a single location increases maintainability and reusability of test cases.

Use case

In the ESB proxy test scenario, the user observes that there are several configuration details related to the backend server included in his test cases. In the synapse configurations he could find the endpoint details as follows;

  

                   

If the user wants to point his ESB proxy service to a different backend service he has to change his test cases. Traditional test development creates overhead when making changes to test cases in the case of environment changes.

It provides comprehensive APIs to retrieve the configurations into tests. Users can use these APIs to delegate configuration details instead of putting them in tests. That facility gives the flexibility of maintaining tests efficiently and adapting tests to environment changes.

Adding configurations to automation.xml

Now the user wants to store the synapse endpoint configurations to the test automation framework’s automation.xml file and retrieve it later in his tests. The WSO2 Test Automation Framework provides a set of default configurations such as platform configurations, user management configurations and security configurations among others. Also it allows the user to store custom configurations details formatted in xml structure (users can decide the structure of their configurations, but it should be formatted in to xml format). So the user can store backend server related configuration details in automation.xml in the following custom format;

  

………..
  
    	
		localhost
        		9000
        		 
            			simpleStockQuoteService
        		
    	
  
………..

Retrieving configurations from automation.xml

WSO2 Test Automation Framework uses the native xpath based xml parsing implementation to retrieve content from the automation.xml file. First, users have to create an AutomationContext object instance in the test. There are multiple constructors in the AutomationContext class and users can use any constructor here to create the instance. Below we use the simplest constructor to do it;

  
AutomationContext automationContext=new AutomationContext();	

Then you can use the AutomationContext instance to obtain configuration details. You have to give the directions to the configuration value which you are going to retrieve, relative to the automation.xml. For that you have to use the xpath of the configuration considering it's xml structure. AutomationContext class provides three different API methods to obtain configuration details. They are as follows;

  1. getConfigurationValue(String XPathToValue) - Retrieve single value of a configuration
  2. getConfigurationNode(String XPathToNode) - Retrieve single configuration node
  3. getConfigurationNodeList(String XPathToNodeList) - Retrieve configuration node list

Users can take one of the above API methods, parsing the xpath by considering automation.xml as the DOM tree to retrieve configurations. Choosing a method from depends on the requirement of the user. If a user wants to obtain a single configuration value (e.g.: port address of the backend server), they should use getConfigurationValue() method.

Now lets see how the user in the ESB proxy test scenario can obtain configuration details of his backend server to the test

Let's consider the following segment of synapse configuration;

  

                   

We can replace the configuration details as follows with the WSO2 Test Automation Framework’s API;

  

String hostname= automationContext.getConfigurationValue
                                                                                  (“//serverConfigurations/axis2Server/host”);
String portAddress=automationContext.
                                                 getConfigurationValue(“//serverConfigurations/axis2Server/port”);

String backendService=automationContext.
getConfigurationValue(“//serverConfigurations/axis2Server/services/service[@name=’testService’]”);
String URI=”https://”+hostname+”:”+portAddress+”/services”+backendService;



                   

Creating custom test environments based on configuration details

Use case

Now the user wants to test his WSO2 ESB proxy service in a clustered environment. For this scenario, the test logic is the same compared to testing for a single WSO2 ESB instance but the testing environment is different. The user has to consider the worker/manager separation and provide appropriate infrastructure details (service URL, backend URL, etc.) to execute the tests. The user wants maintain his test suits with minimal changes to test cases when switching test environments.

WSO2 Test Automation Framework allows users to develop test cases which can be adapted by multiple test environments with minimal effort. Users can store configurations of multiple test environments in a single location (automation.xml file) and create test environments in the run-time based on those configurations. If the user wants to execute his test cases in a new test environment he would only have to store configuration details of the new test environment in the automation.xml file and prepare the appropriate test environment. Then the user can execute his test cases by pointing to the newly prepared test environment.

Now let’s see how we can adapt a simple test case to multiple test environments through an example. Users can store configuration details of multiple environments in automation.xml and create separate environments (using the AutomationContext instance) for each product group under the ‘platform’ section. The ‘ProductGroup’ configuration element holds all configurations of a test environment. We can mention whether the test should be executed in a clustered environment by using the ‘clusteringEnabled’ attribute of a product group. For clustered environments there should be more than one product instance within a product group.

The instance element holds all configurations details of the specific product instance/instances. If we consider an instance node in clustered deployment, that instance represents the clustered setup. If the test environment is not clustered then an instance represents a single product instance.

As an example we will be testing the ESB proxy service in;

  1. Single standalone product instance
  2. Clustered environment where we have load balancer fronted ESB worker and manager instances

The example configuration has two instance;

  1. esbClusteredSetup - Instance to be used to create test environment for clustered environment
  2. esbStandaloneInstance - Instance to be used to create test environment for single ESB product environment

Note: Users have to set the ‘clusteringEnabled’ attribute value to “true” in clustering mode test running and “false” for non-clustered environment tests.

  


 
        	
            	
                	esb.wso2con.com
                	mgt.esb.wso2con.com
            	
            	
                	8243
                	8280
            	
            	
            	
        	

            	
                	localhost
            	
            	
                	9763
                	9443
                	8243
                	8280
            	
            	
            	
        	

Creating test environments in a test case

  • For non-clustered environment
  •   
    AutomationContext nonClusteredEnvironment = new AutomationContext("ESB", "esbStandaloneInstance", TestUserMode.SUPER_TENANT_ADMIN);
    
  • For clustered environment
  •   
    AutomationContext clusteredEnvironment = new AutomationContext("ESB", "esbClusteredSetup", TestUserMode.SUPER_TENANT_ADMIN);
    

    Note: The following example illustrates how to use the above environments in a single test case to show how it adapts to multiple test environments. In this example we check the execution environment using the ‘clusteringEnabled’ attribute of the default product group. The user can make a specific product group the default by setting the ‘default’ attribute to “true” in that product group.

      
    public class ESBProxyAddTestCase  {
     	 
       @BeforeClass(alwaysRun = true)
        protected void uploadCarFileTest() throws Exception {
    
      AutomationContext  customEnvironmentContext=AutomationContext();
    if(customEnvironmentContext.getProductGroup().isClusterEnabled()){
           AutomationContext clusteredEnvironment = new AutomationContext("ESB",     
                       "esbClusteredSetup", TestUserMode.SUPER_TENANT_ADMIN);
    }
    else{
          AutomationContext clusteredEnvironment = new AutomationContext("ESB", "esbClusteredSetup", TestUserMode.SUPER_TENANT_ADMIN);
    }
    
        ProxyServiceAdminClient proxyServiceAdminClient = new  ProxyServiceAdminClient(automationContext.getContextUrls().getBackEndUrl(), 
    automationContext.login());
            	String proxy = "\n" +
                    	"  \n" +
                    	"  \n" +
                    	"  	\n" +
                    	"    	\n" +
                    	"       	\n" +
                    	"          	
    \n" + " \n" + " \n" + " \n" + " \n" + " \n" + ""; OMElement proxyOM = AXIOMUtil.stringToOM(proxy); proxyServiceAdminClient.addProxyService(proxyOM); } }
  • For custom environments (Clustered/non-clustered)
  •   
    AutomationContext clusteredEnvironment = new AutomationContext("ESB", "esbClusteredSetup", TestUserMode.SUPER_TENANT_ADMIN);
    

    Users can use the above test environment in tests for both clustered and non-clustered environments. They have to set a proper value for ‘clusteringEnabled’ attribute (true/false) of the desired product group accordingly before executing the tests. The following example illustrates how we can use this;

      
    
    public class ESBProxyAddTestCase  {
     	 
       @BeforeClass(alwaysRun = true)
        protected void uploadCarFileTest() throws Exception {
    
       AutomationContext automationContext = new AutomationContext("ESB",            TestUserMode.TENANT_USER);
    
        ProxyServiceAdminClient proxyServiceAdminClient = new  ProxyServiceAdminClient(automationContext.getContextUrls().getBackEndUrl(), 
    automationContext.login());
            	String proxy = "\n" +
                    	"  \n" +
                    	"  \n" +
                    	"  	\n" +
                    	"    	\n" +
                    	"       	\n" +
                    	"          	
    \n" + " \n" + " \n" + " \n" + " \n" + " \n" + ""; OMElement proxyOM = AXIOMUtil.stringToOM(proxy); proxyServiceAdminClient.addProxyService(proxyOM); } }

Summary

WSO2 Test Automation Framework fills the gaps between traditional test automation frameworks and modern enterprise test automation requirements providing effective automated test maintainability over multiple test environments.

 

About Author

  • Malintha Adikari
  • Software Engineer
  • WSO2