How to Efficiently Test Service Oriented Architecture

Table of contents

• What is SOA?
• What is SOA testing?
• Importance of SOA testing
• Challenges of SOA testing
• How SOA testing differs from traditional testing
• SOA testers and their testing psychology
• SOA testing lifecycle
• SOA testing tools
• Automation of SOA

What is SOA?

Modern day software solutions hardly work in a standalone mode and must have the ability to securely integrate with disparate data sources and different ecosystems. Handling increasing and changing user expectations can be considered as the key problem that modern enterprise software solutions are trying to resolve.

SOA has emerged as a solution and is considered as an architectural design pattern that can overcome these problems. Although misinterpreted by most people, SOA is not a synonym for web services. However, most SOA-based implementations are built with web services.

Building an enterprise SOA solution from scratch requires a lot of time, effort, and money. Therefore, reusing already developed, third-party components can be cost effective and saves time. Due to demand for such SOA components, software vendors have implemented the required SOA components and some vendors have managed to build entire SOA stacks. WSO2 is one such vendor. The diagram below shows how WSO2 products are used to build a full SOA stack. It will help you get an overall understanding of an SOA solution.

What is SOA testing?

Every software product is supposed to go through a quality assurance cycle to make sure the product that is delivered lives up to its quality standards. Therefore, testing is a major effort required in any software development project.

Most people consider web service testing as SOA testing. However, as mentioned before, SOA is not only about the web services. It is about the overall architecture. In other words, SOA testing should not be restricted to just web services testing.

An SOA solution is generally an integrated set of products that can be a collection of legacy applications, third-party components, or custom developed components, among others. Testers are expected to test not only the individual products and its functionality, but also the overall integrated solution. Therefore, an SOA tester has to consider the bigger picture of SOA for testing, including service providers, service directories, service consumers, communication between SOA components, and authentication providers.

Importance of SOA testing

Businesses today completely depend on their IT systems for their day-to-day business operations, and SOA can be considered as the backbone of the enterprise system. There is no doubt that smooth functioning of enterprise systems is really important for modern businesses and a single error can result in the enterprise losing large sums of money and damaging the business’ brand.

SOA solutions are really complex in nature and due to the challenges, which will be discussed later in this article, the testing phase of an SOA solution is crucial.

Challenges of SOA testing

Testing is undoubtedly a challenging task and can become increasingly challenging due to reasons like growing customer expectations, rapid changes in systems, limited time frames, etc.

The following points can be identified as some of the more unique challenges when in SOA testing:

• Complex and heterogeneous nature of SOA solutions
• SOA testing being more data driven than traditional testing
• The need to check product compliance against complex technical specs and standards
• The larger set of testing combinations with different types of products that can be integrated
• Difficulty to simulate testing environments
• Difficulty to reproduce issues
• The need to focus on complete business use cases instead of just a product feature
• High technical competency required from testers
• The need for more attention on load and performance testing
• Inclusion of different vendor products
• Limited time for testing
• Continuous growth of the platform with new features, new products, etc.

How SOA testing differs from traditional testing

When testing normal applications, testers highly depend and interact on the respective application GUIs. However, SOA testers do not have that luxury as most SOA components do not come with GUIs and normally expose programmatic interfaces like APIs and WSDLs.

Normal web applications can be tested on general user perspective as, most times, real users of considered applications are non-technical. Yet, when it comes to SOA testing, end-users of most SOA solutions/components are highly technical people like solution architects, CIOs, and software developers. Therefore, testers need to test the applications from a developer point of view. SOA testers need to possess expertise in both the business and technology domains to effectively test it.

SOA testing requires both service consumers and service providers to be tested. The SOA platform mostly provides just the section for the service providers, so the testers need to implement matching SOA client apps in order to perform testing. As a result, the testers need to play the role of a developer as well.

Given rapid release cycles, less time is allocated for testing, so it is very important to automate all possible testing scenarios. GUI-based automation plays a major role when automating normal web applications; however, for an SOA solution, automation needs to be done through programmatic interfaces.

In traditional testing, the tester has to wait until the developers complete the features of the system; however, in SOA testing, they can start testing at earlier stages by creating similar mock services.

To highlight the exact differences between traditional testing and SOA testing, we have used the Message Store and Message Processor JMS Use Case scenario that is documented in the WSO2 ESB documentation section.

Traditional approach

The tester will publish the messages through the proxy service and checks if he/she receives the expected response at the Reply Sequence. The tester only executes simple test cases and generates the test results depending on them.

SOA approach

Here, the tester needs to test all the scenarios listed below.

• Make sure the Reply Sequence receives the respective response message accordingly
• Find out how the system behaves in case of Backend Service and Message Store down time
• Find out how the system behaves when there are high loads
• Check whether there is any message duplication, modification, or loss in the system
• Find out how the system behaves with a slow backend service
• Find out how the system behaves to the different types of message formats sent through the proxy service
• Test the individual Message Store and the Message Forwarding Processor feature

SOA testing strategies

As discussed above, the nature of SOA-based solutions call for different testing strategies and a different toolset.

SOA testers need to have a specific set of skills and a different mindset because they are required to identify a different approach.

There are a few identified approaches for SOA testing. They include bottom-up testing, top-down testing, and system testing. Testers can use any or all of these approaches. Most successful SOA testing projects use all approaches and place emphasis on one approach based on the requirements of the architecture.

SOA testers and their testing psychology

The following qualities are generally considered essential for a software tester. SOA testers should posses the same qualities. These include

• Attention to detail and keen observation
• Ability to detect possible problems quickly
• Passion and willingness to break the software systems
• Understand the product and integration of its parts
• Be able to look into issues from the perspective of customers/users
• Willingness to object to and question every aspect of the software
• Willingness to complete everything perfectly and in a well-ordered manner

SOA testers need to specifically possess certain unique qualities and approaches, such as as the following:

• Test the application with a developer-oriented mindset and not from the traditional user perspective
• Have a wider knowledge of all aspects in IT from OS administration and database administration to programming languages and networking
• Have fair knowledge in software development
• Have a thorough understanding of SOA paradigms

SOA testing lifecycle

In the traditional software testing lifecycle, the following phases are executed and each phase has its own entry criteria and deliverables. These processes may vary according to the software development lifecycle that is being used as well as management decisions.

1. Requirements/design review
2. Test planning
3. Test designing
4. Test environment setup
5. Test execution
6. Test reporting

When performing SOA testing, a similar process can be followed, but testers need to put more effort into configuring required test environments due to the complex nature of SOA solutions.

During the test execution phase, SOA testers need to put in more effort into and focus more attention on the areas mentioned in the sections below.

Performance

Any middleware platform should perform swiftly to support the business transactions. Therefore, identifying performance bottlenecks is a key aspect that an SOA tester would need to focus on.

Load

With the rapid growth of technology, enterprise applications require the capacity to handle more data and more transactions. As a result, it is very important to consider load testing.

QoS

Each SOA service has a Quality of Service (QoS) associated with it. Some of the key QoS elements are security (i.e. authentication and authorization), reliable messaging, caching, and throttling. By using different policies, who can access what, when and how often can be restricted. The testers will have to test for all these factors.

Furthermore, SOA testers need to make sure that the SOA solution and its components work according to the standard specification. For example, with regard to security, components should follow standards such as Security Assertion Markup Language (SAML) and WS-Security.

Reliability

SOA has a loosely coupled nature and several messages usually need to be exchanged between the service consumers and the service providers to accomplish a business activity. As a result, it is extremely important to ensure that the messages flow between the systems as expected. SOA testers need to make sure that the system behaves in a reliable manner.

Clustering

Due to the increasing transactions and data handling loads, a single SOA component may not be able to handle the expected loads. Hence, those components are required to run with multiple nodes to support the required scalability. In addition, for high availability and failover scenarios, running an SOA solution with multiple nodes is important. Therefore, testers have to make sure that the required components run smoothly in clustered environments.

Integration

As we highlighted in previous sections of the article, a key advantage of SOA is its ability to integrate different kinds of products. The tester will have to focus on possible integration scenarios and make sure that the SOA solution integrates accurately.

Platform support

SOA platform components are supposed to run on different operating systems. Ensuring that the products are running on all major operating systems is also a key part of the SOA testing cycle.

SOA testing tools

Having a proper tool set is very important for SOA testing. Effective and relevant tools will maximize productivity and produce accurate outcomes with the expected quality.

When selecting a testing tool for SOA testing the following factors should be considered:

• The ability to generate request messages automatically
• The ability to validate responses using assertions
• QoS-enabled service invocation
• Service simulation
• Support for multiple transport protocols
• Multiple message support

Testing tools

SoapUI - SoapUI is an open-source tool that can be used for web services testing. In addition to functional testing, SoapUI can also be used for performance testing. One of the advantages of using SoapUI is that you can use it to perform security testing.

Apache Jmeter - JMeter is an open-source tool that can be used for analyzing and measuring performance and performing REST/SOAP invocations. This tool can be used to automate most user stories. It allows the tester to run the same test with different users and also has the ability to perform parameterization. One of the drawbacks of this tool is that it cannot be used to automate security-related scenarios.

cURL - cURL is a command line tool that is used to send or retrieve information with the use of URL syntax. This tool supports many internet protocols such as HTTP, HTTPS, FTP, IMAP, and many more. cURL is not heavily used for load testing and is mainly used for functional testing.

Refer to WSO2 Library article about SOA tools to learn more about SOA testing tools.

Monitoring tools

JConsole - JConsole is a graphical monitoring tool that is used to monitor Java Virtual Machine (JVM) and Java applications, both on local and on remote machines.

JProfiler - The JProfiler can be used to identify performance bottlenecks, pin down memory leaks, and understand threading issues.

Traditional tools that are used with the testing life cycle can also be used for SOA testing. SOA testers can use tools like Microsoft Test Manager and TestLink for test case management. Other tools like Atlassian JIRA, Bugzilla, Mantis, and Redmine can be used for issue tracking and issue management.

Test automation of SOA

Maintaining the quality of the SOA platform is a tedious job. Increasing the manual test team is not practical or feasible in the long run. Therefore, automating the SOA as much as possible is the only solution.

However, SOA automation is quite different from normal web application automation. In web automation, testers depend heavily on user interfaces; however, in SOA automation, testers have to focus on programmatic interfaces and services.

The following areas can be identified as key areas that require consideration for automation.

• Test environment setup/clustering
• Unit/module testing
• Product level feature automation
• Integration scenario automation

Summary

Having an idea about SOA and its uniqueness will significantly help to effectively carry out SOA testing. Throughout this article, we have explained the nature of an SOA solution and how it differs from a traditional software solution. With that understanding, it’s also key to identify the unique challenges in SOA testing and the required approaches to overcome these challenges.