How to Write Connector Integration Tests Using WSO2 Test Automation Framework
- Dimuthu De Lanerolle
- Software Engineer - WSO2
Introduction
We assume readers have basic knowledge on the TestNG framework. You can refer to TestNG documentation for the initial knowledge required. To become more familiar with WSO2 Test Automation Framework and to follow generic rules on writing integration tests with WSO2 TAF, refer to WSO2 TAF documentation.
In this article, we analyze the basic scenario of using Facebook connectors to write a sample integration test to get event details of a posted event in a given Facebook profile.
What are connectors?
A connector allows you to interact with a third-party product’s functionality and data from your message flow.
About WSO2 ESB connectors
WSO2 ESB allows you to create your own connectors or use pre-implemented connectors, which are capable of allowing your message flows to connect and interact with third-party services, such as Facebook,Twitter, Twilio, Google Spreadsheet, etc.
For example, let’s think about a situation where you have enabled Twitter and Google Spreadsheet connectors in your ESB instance; your message flow could receive requests containing a user's Twitter name and password, log into the user's Twitter account, get a list of the user's followers, and write that information to a Google spreadsheet. Each connector provides a set of operations. After adding the required connector to your ESB instance, you can start invoking these operations inside your test class.
Click on this link below to download some pre-implemented connectors.
How to write a connector integration test
We will now illustrate some key steps involved in tackling this problem.
To start with, you need to create a module in your test location, e.g. you can start writing your tests in the following location.
…./home/xxx/xxxx/esb-connectors/
For this illustration we will consider a situation where your ESB instance interacts with the Facebook connector.
1. You can clone the WSO2 ESB connector module from the following github HTTP clone URL https://github.com/wso2-dev/esb-connectors.git
2. Now find “Facebook” module inside esb-connectors
Build the connector and place the generated facebook.zip file in xxxx/esb-connectors/facebook/src/test/resources/artifacts/ESB/connectors
Here are the basic dependencies you need have inside the ....esb-connectors/facebook/pom.xml file.
Note: You might need to replace the versions of the dependencies listed here in accordance with the WSO2 ESB version you are running (these dependency versions will work with WSO2 ESB 4.8.1 only).
org.wso2.carbon.automation org.wso2.carbon.automation.engine ${automation.framework.version} org.wso2.carbon.automation org.wso2.carbon.automation.test.utils ${automation.framework.version} org.wso2.carbon.automation org.wso2.carbon.automation.extensions ${automation.framework.version} org.wso2.carbon org.wso2.carbon.integration.common.extensions ${common.version} org.wso2.carbon org.wso2.carbon.integration.common.admin.client ${common.version} org.wso2.carbon org.wso2.carbon.mediation.library.stub ${stub.version} test javax.servlet servlet-api org.wso2.carbon org.wso2.carbon.proxyadmin.stub ${proxyadmin.stub.version} org.wso2.carbon org.wso2.carbon.mediation.initializer ${stub.version} org.apache.synapse synapse-core ${synapse.core.version} org.wso2.carbon org.wso2.carbon.mediation.configadmin.stub ${stub.version} 4.3.1-SNAPSHOT 4.2.0 2.1.1-wso2v7 4.3.0-SNAPSHOT 4.2.1
Note: There are several points to ponder when writing connector-related test classes. We will now list down each and you should carefully read the notes below as these will be practically used inside the sample test class we will be writing soon.
1. Your ESB distribution should contain the following entries in its axis2.xml
You can find the axis2.xml in wso2esb-4.8.1/repository/conf/axis2 inside the distribution.
2. For this test class scenario, we will have to create a new module “tests-common” form the esb-connectors module level and create another module and name it “admin-clients” in-order to place product specific admin clients. In our case we will add a few source classes, namely “ProxyAdminClients.java” and “SynapseConfigAdminClient.java” to this package so that every test class inside our “Facebook” connector module can directly invoke methods inside these classes. We will look into more details in this regard at a later stage.
Note: To refer more on ProxyAdminClients.java and SynapseConfigAdminClient.java refer to the links below that contain sample codes for these classes.
[1]ProxyServiceAdminClient.java
[2]SynapseConfigAdminClient.java
3. create another module called “integration-test-utils” inside “tests-common”. In-order to maintain consistency and convenience between test classes inside numerous test modules we will implement a generic base test class that contains all the common methods that in most cases every test class we add to our test module might be using. We will name it “ESBIntegrationConnectorBaseTest.java” class and as mentioned most of the common methods for the whole module will be readily available to other tests classes to extend and carry out their work. For instance our sample test class “FacebookConnectorIntegrationTest .java”
class will extend this “ESBIntegrationConnectorBaseTest.java” class at the first place so that we can waive the burden of such tedious, repetitive workload such as initialization of AutomationContext objects, writing requests, and reading responses. This will save our time in repeating many code snippets every time we add a new test class to our module.
Given below is the structure.
/esb-connectors/
|--> tests-common
| |--> admin-clients
| |--> ProxyServiceAdminClient.java
| |--> SynapseConfigAdminClient.java
|
| |--> integration-test-utils
| |--> ESBIntegrationConnectorBaseTest.java
| |--> ESBTestCaseUtils.java
|
|--> facebook/src/test/
| java
|-->org.wso2.carbon.connector.integration.test.facebook
|--> FacebookConnectorIntegrationTest.java
| resources
| artifacts
|-->AXIS2
|-->ESB
|.....
|--> connectors
|--> facebook.zip
|.....
| axis2config
| client.modules
| keystores
| security
| automation.xml
| automationSchema.xsd
| emma.properties
| filters.txt
| instrumentation.txt
| testng.xml
To begin with, as mentioned above, navigate to integration-test-utils package and create a new java class. We will name it ESBIntegrationConnectorBaseTest.java. This is the class we should place methods that are common to almost all test classes.
In most cases, it is inevitable that we create an Automation context object for our test scenarios. Automation context object is more of a custom runtime environment that suits running your tests. WSO2 Test Automation Framework will allow you to create an AutomationContext object in accordance with the provided parameters given at the initial stage of the test.
Implement a protected init() method and create an instance from the AutomationContext.java class by passing relevant input parameters to the constructor of the AutomationContext class.
new AutomationContext("ESB", TestUserMode.SUPER_TENANT_ADMIN);
Here “ESB” is an already defined productGroup name in the automation.xml file.
To learn more about the automation.xml file and its capabilities refer to the below link that describes the automation.xml in depth.
[1] Automation.xml File Description
Refer to the below link for automation.xml file.
[1] automation.xml
Moreover, note that there are several types of constructors readily available in the AutomationContext.java class enabling you to define the range of automation instances as per your requirement.
In addition, you need to implement a login() method to perform the login operation to the ESB server and obtain a session cookie. Given below is a sample code snippet for a login method and you can create your own using this as a foundation.
public String login() throws IOException,
LoginAuthenticationExceptionException, XPathExpressionException,
XMLStreamException, SAXException, URISyntaxException {
LoginLogoutClient loginLogoutClient = new LoginLogoutClient(automationContext);
return loginLogoutClient.login();
}
Moreover, you can add similar common methods to the ESBIntegrationConnectorBaseTest.java class that you might need when writing your test scenarios. Note how to derive backend URLs, usernames, and passwords.
Now let’s look into more details relating to the starting of writing our test. Create your own test class inside the “Facebook” module. We will name this class FacebookConnectorIntegrationTest .java . Now, as mentioned in the above, you need to extend ESBIntegrationConnectorBaseTest.java class.
public class FacebookConnectorIntegrationTest extends ESBIntegrationConnectorBaseTest
{ ….}
@BeforeClass(alwaysRun = true)
public void setEnvironment() throws Exception {...}
Since we have set alwaysRun = true this configuration method will run regardless of what group it belongs to
The init(..) method in setEnvironment(..) will initialize the environment essential to run our tests. This is the place where we create and initialize our AutomationContext object. In addition, we can initialize some service variables and instances at the first place before proceeding with the actual test case scenarios.
Check whether you have connector configuration files under
.../facebook/src/test/resources/artifacts/ESB directory.
Make sure the existence of the connector (facebook.zip), facebook.properties configuration file and the facebook.xml proxy file in resources directory.
E.g. …./esb-connectors/facebook/src/test/resources/artifacts/ESB/connectors/facebook.zip
Skim through the properties mentioned in facebook.properties file. As our test case will basically focus on adding a proxy to the esb server and get a particular event details from the a facebook account we will need to introduce some property tags to facebook.properties file. The usage of facebook.properties file is to store “Facebook” connector specific configurations enabling us to customize our code.
E.g. facebook.properties
# proxy folder proxyDirectoryRelativePath=/../src/test/resources/artifacts/ESB/config/proxies/facebook/ # Folder for of the Rest Request files requestDirectoryRelativePath=/../../../../../../src/test/resources/artifacts/ESB/config/restRequests/facebook/ # Folder for the resources to be used resourceDirectoryRelativePath=/../../../../../../src/test/resources/artifacts/ESB/config/resources/facebook/ # Access Token accessToken=CAACEdEose0cBAERyQe4ow7IJib9SFkVZBPtasVc1yovfeJNTK1N5RPqYcsm2JXELw819E9GfWYiEYlOA350JT3hyZBaDihfLm9IqScGJPEfmKLqfgpph9UBRpmOi2tUXRgAP8E8jHbzQeQctWjYlo1IwSJnCVAqcAsZCzEnM3WTkuIb251GfA06dZB6qGbiKSqZA1EhbnrQZDZD # Third party user to create invitation and tag photo; must be a friend. friendId=1236947282 # User profile ID userId=100007639237322 # The message text of the notification in method PublishNotification template=This is Application Notification # Page Access token pageAccessToken=CAACEdEose0cBAFZBmMiJ7SmgKZBe11TeZBn4ZC5CVFFC2IQDZCPfUf1ZBboDllC2iZCyly1wOu0XHiuGtBUvqO2j1XLur3RkhlnjvMnHtymwrZBgXxoU1pyubSXgSqZAryMvJMaJt0xMf7ZCZB2iJCjEL1xORXPQUwpMWQdKpK4l0VKSnUXhTHV2giQXPxShOobmfkwlEkY8WqJoAZDZD # The page Id which received 50 likes. pageId=473300532775365 # General Description to be used description=Connector Development # General Message to be used message=Connector Development Message #Event ID eventId=630793950344316 # third party user to be banned/unbanned needs to be added to Application appUserId=100008133212722 # Application Id appId=491694797617714 # update page settings (must be a boolean value). value=false # Url of the facebook Graph API apiUrl=https://graph.facebook.com/
Creating an event
Follow these steps for adding an event related properties to facebook.properties file.
1. Create a new Facebook account (or you may use an existing Facebook account known to you for testing purposes)
Note: Your account should be a verified developer account.
Access your Facebook account using your credentials.
2.Obtain an "id" using me/?fields=id in "Graph Explorer" (https://developers.facebook.com/tools/explorer) and copy in to userId in facebook.properties file.
3. Navigate to homepage of your Facebook account and click on “Events”. You should be able to see the facebook events page. Click the Create Event button and now in the “Create New Event” dialog box fill the relevant details and finally click create button. You have successfully added an event to your event list.
You can view the Event ID from the url.
E.g. https://www.facebook.com/events/630793950344316/?ref_dashboard_filter=upcoming
From the above URL, our Event ID would be 630793950344316. Make sure to add another entry to facebook.properties file indicating related details of the event we created.
E.g.
#Created Event ID
eventId=630793950344316
# Name of the event i
eventName=Connector Development Review
Uploading the connector zip file
As mentioned, make sure to place your facebook.zip file in the ../esb-connectors/facebook/src/test/resources/artifacts/ESB/connectors directory.
Refer to the code snippet for ESBIntegrationConnectorBaseTest.java to find the usage of the Facebook connector.
Testing scenario
We will create a proxy service in the ESB server, and with this proxy service, we will call the api-endpoint of the event list from the Facebook account and verify its details.
Running the test class
Add following xml elements to testng.xml file.
Resources
facebook.xml file (xx/esb-connectors/facebook/src/test/resources/artifacts/ESB/synapseconfig/facebook)
{$ctx:apiUrl} {$ctx:accessToken} {$ctx:connection} {$ctx:fields} {$ctx:eventId}
FacebookConnectorIntegrationTest .java
package org.wso2.carbon.connector.integration.test.facebook;
import integrationtestutils.ESBIntegrationConnectorBaseTest;
import org.json.JSONException;
import org.json.JSONObject;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import static org.testng.AssertJUnit.assertEquals;
public class FacebookConnectorIntegrationTest extends ESBIntegrationConnectorBaseTest {
private Map esbRequestHeadersMap = new HashMap();
private Map apiRequestHeadersMap = new HashMap();
@BeforeClass(alwaysRun = true)
public void setEnvironment() throws Exception {
init("facebook");
esbRequestHeadersMap.put("Accept-Charset", "UTF-8");
esbRequestHeadersMap.put("Content-Type", "application/json");
apiRequestHeadersMap.put("Accept-Charset", "UTF-8");
apiRequestHeadersMap.put("Content-Type", "application/x-www-form-urlencoded");
}
@Test(groups = {"wso2.esb"}, description = "getting facebook event by event ID")
public void testGetEventDetailsWithMandatoryParameters() throws IOException, JSONException {
esbRequestHeadersMap.put("Action", "urn:getEventDetails");
String apiEndPoint =
connectorProperties.getProperty("apiUrl") + connectorProperties.getProperty("eventId")
+ "?access_token=" + connectorProperties.getProperty("accessToken");
RestResponse esbRestResponse =
sendJsonRestRequest(proxyUrl, "POST", esbRequestHeadersMap, "esb_getEventDetails_mandatory.txt");
RestResponse apiRestResponse = sendJsonRestRequest(apiEndPoint, "GET", apiRequestHeadersMap);
assertEquals(esbRestResponse.getBody().get("start_time"), apiRestResponse.getBody().get("start_time"));
assertEquals(esbRestResponse.getBody().get("name"), apiRestResponse.getBody().get("name"));
assertEquals(esbRestResponse.getBody().get("id"), apiRestResponse.getBody().get("id"));
}
}
package integrationtestutils;
import org.apache.axiom.om.OMElement;
import org.apache.axis2.context.ConfigurationContext;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.json.JSONException;
import org.json.JSONObject;
import org.wso2.carbon.authenticator.stub.LoginAuthenticationExceptionException;
import org.wso2.carbon.automation.engine.context.AutomationContext;
import org.wso2.carbon.automation.engine.context.TestUserMode;
import org.wso2.carbon.automation.engine.frameworkutils.FrameworkPathUtil;
import org.wso2.carbon.automation.test.utils.axis2client.ConfigurationContextProvider;
import org.wso2.carbon.connector.integration.test.facebook.RestResponse;
import org.wso2.carbon.integration.common.utils.LoginLogoutClient;
import org.wso2.carbon.mediation.library.stub.MediationLibraryAdminServiceStub;
import org.wso2.carbon.mediation.library.stub.upload.MediationLibraryUploaderStub;
import org.wso2.carbon.mediation.library.stub.upload.types.carbon.LibraryFileItem;
import org.xml.sax.SAXException;
import javax.activation.DataHandler;
import javax.xml.stream.XMLStreamException;
import javax.xml.xpath.XPathExpressionException;
import java.io.*;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URISyntaxException;
import java.net.URL;
import java.nio.charset.Charset;
import java.rmi.RemoteException;
import java.util.*;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import static org.wso2.carbon.integration.common.admin.client.utils.AuthenticateStubUtil.authenticateStub;
public class ESBIntegrationConnectorBaseTest {
private static final Log log = LogFactory.getLog(ESBIntegrationConnectorBaseTest.class);
private static final float SLEEP_TIMER_PROGRESSION_FACTOR = 0.5f;
private AutomationContext automationContext;
private MediationLibraryUploaderStub mediationLibUploadStub;
private MediationLibraryAdminServiceStub adminServiceStub;
protected Properties connectorProperties;
protected String proxyUrl;
private String repoLocation;
private String pathToRequestsDirectory;
protected String pathToResourcesDirectory;
protected String getBackendURL() throws XPathExpressionException {
return automationContext.getContextUrls().getBackEndUrl();
}
protected String getServiceURL() throws XPathExpressionException {
return automationContext.getContextUrls().getServiceUrl();
}
protected void init(String connectorName) throws Exception {
automationContext = new AutomationContext("ESB", TestUserMode.SUPER_TENANT_ADMIN);
ConfigurationContextProvider configurationContextProvider = ConfigurationContextProvider.getInstance();
ConfigurationContext cc = configurationContextProvider.getConfigurationContext();
mediationLibUploadStub =
new MediationLibraryUploaderStub(cc, getBackendURL() + "MediationLibraryUploader");
authenticateStub("admin", "admin", mediationLibUploadStub);
adminServiceStub =
new MediationLibraryAdminServiceStub(cc, automationContext.getContextUrls().getBackEndUrl() + "MediationLibraryAdminService");
authenticateStub("admin", "admin", adminServiceStub);
if (System.getProperty("os.name").toLowerCase().contains("windows")) {
repoLocation = System.getProperty("connector_repo").replace("\\", "/");
} else {
repoLocation = System.getProperty("connector_repo").replace("/", "/");
}
//new ProxyServiceAdminClient(automationContext.getContextUrls().getBackEndUrl(), login());
String connectorFileName = connectorName + ".zip";
uploadConnector(repoLocation, mediationLibUploadStub, connectorFileName);
byte maxAttempts = 3;
int sleepTimer = 30000;
for (byte attemptCount = 0; attemptCount sendJsonRestRequest(String endPoint, String httpMethod,
Map headersMap) throws IOException, JSONException {
return this.sendJsonRestRequest(endPoint, httpMethod, headersMap, null, null);
}
private Properties getConnectorConfigProperties(String connectorName) {
String connectorConfigFile;
try {
connectorConfigFile =
FrameworkPathUtil.getSystemResourceLocation() + File.separator + "artifacts" + File.separator
+ "ESB" + File.separator + "connector" + File.separator + "config" + File.separator
+ connectorName + ".properties";
File connectorPropertyFile = new File(connectorConfigFile);
InputStream inputStream = null;
if (connectorPropertyFile.exists()) {
inputStream = new FileInputStream(connectorPropertyFile);
}
if (inputStream != null) {
Properties prop = new Properties();
prop.load(inputStream);
inputStream.close();
return prop;
}
} catch (IOException ignored) {
log.error("automation.properties file not found, please check your configuration");
}
return null;
}
private void uploadConnector(String repoLocation, MediationLibraryUploaderStub mediationLibUploadStub,
String strFileName) throws MalformedURLException, RemoteException {
List uploadLibraryInfoList = new ArrayList();
LibraryFileItem uploadedFileItem = new LibraryFileItem();
uploadedFileItem.setDataHandler(new DataHandler(new URL("file:" + "///" + repoLocation + "/" + strFileName)));
uploadedFileItem.setFileName(strFileName);
uploadedFileItem.setFileType("zip");
uploadLibraryInfoList.add(uploadedFileItem);
LibraryFileItem[] uploadServiceTypes = new LibraryFileItem[uploadLibraryInfoList.size()];
uploadServiceTypes = uploadLibraryInfoList.toArray(uploadServiceTypes);
mediationLibUploadStub.uploadLibrary(uploadServiceTypes);
}
protected String getProxyServiceURL(String proxyServiceName) throws XPathExpressionException {
return automationContext.getContextUrls().getServiceUrl() + "/" + proxyServiceName;
}
protected RestResponse sendJsonRestRequest(String endPoint, String httpMethod,
Map headersMap, String requestFileName, Map parametersMap)
throws IOException, JSONException {
HttpURLConnection httpConnection =
writeRequest(endPoint, httpMethod, RestResponse.JSON_TYPE, headersMap, requestFileName, parametersMap);
String responseString = readResponse(httpConnection);
RestResponse restResponse = new RestResponse();
restResponse.setHttpStatusCode(httpConnection.getResponseCode());
restResponse.setHeadersMap(httpConnection.getHeaderFields());
if (responseString != null) {
JSONObject jsonObject = null;
if (isValidJSON(responseString)) {
jsonObject = new JSONObject(responseString);
} else {
jsonObject = new JSONObject();
jsonObject.put("output", responseString);
}
restResponse.setBody(jsonObject);
}
return restResponse;
}
private boolean isValidJSON(String json) {
try {
new JSONObject(json);
return true;
} catch (JSONException ex) {
return false;
}
}
private HttpURLConnection writeRequest(String endPoint, String httpMethod, byte responseType,
Map headersMap, String requestFileName, Map parametersMap)
throws IOException {
String requestData = "";
if (requestFileName != null && !requestFileName.isEmpty()) {
requestData = loadRequestFromFile(requestFileName, parametersMap);
} else if (responseType == RestResponse.JSON_TYPE) {
requestData = "{}";
}
OutputStream output = null;
URL url = new URL(endPoint);
HttpURLConnection httpConnection = (HttpURLConnection) url.openConnection();
httpConnection.setRequestMethod(httpMethod);
for (String key : headersMap.keySet()) {
httpConnection.setRequestProperty(key, headersMap.get(key));
}
if (httpMethod.equalsIgnoreCase("POST")) {
httpConnection.setDoOutput(true);
try {
output = httpConnection.getOutputStream();
output.write(requestData.getBytes(Charset.defaultCharset()));
} finally {
if (output != null) {
try {
output.close();
} catch (IOException logOrIgnore) {
log.error("Error while closing the connection");
}
}
}
}
return httpConnection;
}
private String loadRequestFromFile(String requestFileName, Map parametersMap) throws IOException {
String requestFilePath;
String requestData;
requestFilePath = pathToRequestsDirectory + requestFileName;
requestData = getFileContent(requestFilePath);
Properties prop = (Properties) connectorProperties.clone();
if (parametersMap != null) {
prop.putAll(parametersMap);
}
Matcher matcher = Pattern.compile("%s\\(([A-Za-z0-9]*)\\)", Pattern.DOTALL).matcher(requestData);
while (matcher.find()) {
String key = matcher.group(1);
requestData = requestData.replaceAll("%s\\(" + key + "\\)", prop.getProperty(key));
}
return requestData;
}
private String readResponse(HttpURLConnection con) throws IOException {
InputStream responseStream = null;
String responseString = null;
if (con.getResponseCode() >= 400) {
responseStream = con.getErrorStream();
} else {
responseStream = con.getInputStream();
}
if (responseStream != null) {
StringBuilder stringBuilder = new StringBuilder();
byte[] bytes = new byte[1024];
int len;
while ((len = responseStream.read(bytes)) != -1) {
stringBuilder.append(new String(bytes, 0, len));
}
if (!stringBuilder.toString().trim().isEmpty()) {
responseString = stringBuilder.toString();
}
}
return responseString;
}
private String getFileContent(String path) throws IOException {
String fileContent = null;
BufferedInputStream bfist = new BufferedInputStream(new FileInputStream(path));
try {
byte[] buf = new byte[bfist.available()];
bfist.read(buf);
fileContent = new String(buf);
} catch (IOException ioe) {
log.error("Error reading request from file.", ioe);
} finally {
if (bfist != null) {
bfist.close();
}
}
return fileContent;
}
protected RestResponse sendJsonRestRequest(String endPoint, String httpMethod,
Map headersMap, String requestFileName) throws IOException, JSONException {
return this.sendJsonRestRequest(endPoint, httpMethod, headersMap, requestFileName, null);
}
public String login() throws IOException,
LoginAuthenticationExceptionException, XPathExpressionException,
XMLStreamException, SAXException, URISyntaxException {
LoginLogoutClient loginLogoutClient = new LoginLogoutClient(automationContext);
return loginLogoutClient.login();
}
}
Summary
This article provided a step-by-step guide on our testing scenario. This article can be used as a foundation and guide for users to implement different testing scenarios using the Facebook connector.