Introducing WSO2 Micro Integrator 1.2
- Sasikala Kottegoda
- Software Engineer - WSO2
Executive Summary
Integration has become the major building block in today’s software industry because with integration, it is possible to create fully-functional, robust solutions. WSO2 Micro Integrator is a solution that helps to design complex integration flows using a user-friendly drag-and-drop designer panel. It can support both on-premise and microservice applications with a bunch of powerful features for debugging, testing, and CI/CD implementations. In this article, we’ll be exploring the new features and improvements shipped in WSO2 Micro Integrator 1.2 such as task coordination, readiness probe, swagger generation for data services, and more.
Introduction
WSO2 Micro Integrator 1.2 brings you the best of both worlds of centralized deployments and containers. The renowned synapse engine brings in all the required integration capabilities. Its adaptable nature makes it easier to use for microservices. This release specifically focuses on three aspects:
- The distributed consensus in centralized deployments
- Enhanced native support for cloud deployments
- Improved developer experience
New Features
These are the new features that are shipped with WSO2 Micro Integrator 1.2.
Scalability in Centralized Deployments
Previous releases of WSO2 Micro Integrator were intended to work as individual units that perform independent tasks. They did not have the capability of having a distributed consensus at work. The latest release brings in the capability of having a distributed consensus for tasks carried out among the nodes and hence brings up the concept of a cluster by means of an RDBMS database. Feature 1 explains this in more detail.
Enhanced Native Support for Cloud Deployments
This release comes with a number of features and improvements for containerized deployments, such as the readiness probe implementation, improved vault implementation along with Docker/K8s secrets support, enhanced environment variable support, Helm charts, etc. It should be noted that the Docker images for the Integrator Dashboard are now available in the Docker Hub along with this release.
Improved Developer Experience
A number of improvements have been made in order to uplift the developer experience. An enhanced experience for engaging security along with WSO2 Integration Studio, support for hot deployment of synapse artifacts, swagger definition generation for data services, user management with JDBC user stores, and accessing log files via the Integration Dashboard is the most prominent out of those.
Feature 1: Task Coordination in a Distributed Environment
This feature provides the capability to have coordination among tasks when WSO2 Micro Integrator is deployed in a centralized environment. When the deployment is clustered, the tasks that are running in the server (e.g., Scheduled Tasks, Inbound Endpoints Tasks, and Message Processor Tasks) need to have coordination among themselves to ensure high availability and consistency.
The coordination is achieved in WSO2 Micro Integrator using an RDBMS-based consensus algorithm. Every node in the cluster communicates via the centralized database to achieve consensus on the node in which the task needs to be performed. The leader node will have the monopoly to select the task node based on the configurations.
Figure 1: High Level architecture of a MI cluster
Feature 2: Readiness Probe for Health Checking
The WSO2 Micro Integrator runtime has been developed to cater to the needs of both centralized and containerized deployments. As an enterprise server, there should be some cues indicating whether the server has started successfully and healthy. In a centralized environment, this is straightforward. With hot deployment enabled, you just have to ensure whether the server started successfully. In a containerized environment, we expect the server to be immutable. Once you burn an image with CApps included, you will never change what it is inside the server. Hence, it is important to check whether the deployed CApps started successfully or not. Most of the time, containers are used with a container orchestration platform like Kubernetes. Starting and killing containers happen frequently. So, when a container says that it is ready to serve requests, it is vital that the CApps deployed inside have started without any faults.
With this release, we have introduced a Readiness Probe, which indicates that the server has started and the CApps are deployed successfully. This feature has two behaviors depending on whether the deployment is centralized or containerized. If the deployment is centralized (hot deployment is enabled), the probe will give a ready status when the server starts successfully. If the deployment is containerized, the probe will give a ready only status if all CApps are deployed successfully during server startup. If there are faulty CApps, the probe will return the list of fault CApps.
Figure 2: Sample response for the Readiness probe invocation
Feature 3: Hot Deployment Support
Previous releases primarily focused on containerized deployments of the server; hence, the requirement of immutability. However, having to restart the server for each artifact modification is time-consuming and the developer experience would be better if dynamic updates of deployment artifacts are supported.
The release of WSO2 Micro Integrator 1.2 supports hot deployment of synapse artifacts (XML), data services (DBS), and carbon applications (CAR). It is enabled by default in the Micro Integrator distribution in order to enhance the developer experience. However, this is disabled in the base Docker image of WSO2 Micro Integrator to honor the immutable nature of containerized deployments.
Feature 4: Swagger Generation for Data Services
WSO2 Micro Integrator 1.2 brings in the capability of generating a corresponding swagger definition automatically for data service with resources (RESTful resources defined in that data service). Also, sometimes you may need to change the automatically generated swagger definition with custom edits. Or, you might already have a swagger definition that you need to expose through the data service. In that case, with this feature, a user can add those definitions to the Micro Integrator's registry and publish it without publishing the automatically generated swagger definition.
You can access this Swagger document by suffixing the service URL with ?swagger.json or ?swagger.yaml as shown below.
JSON Format
https://localhost:8290/services/
The Sample Swagger Definition generated in JSON format is given below.
Figure 3: Sample swagger definition for a dataservice in JSON
YAML Format
https://localhost:8290/services/
The Sample Swagger Definition generated in YAML format is given below.
Figure 4: Sample swagger definition for a dataservice in YAML
Feature 5: User Management With External User Stores
WSO2 Micro Integrator 1.2 brings in the capability of managing users in user stores you plug in externally. This allows new users to simply configure an RDBMS database or an LDAP and create/delete/retrieve users to and from the database, whereas older users can plug in their WSO2 JDBC/LDAP user store that is already present and modify the users in it.
This feature is supported via the Integrator Dashboard, the command-line tool, and the management API so that users can choose the desired mode of modifying the user store. The operation of modifying users is safely secured so that only the users with admin privileges can perform any action on the user stores.
Figure 5: Dashboard view for users
Feature 6: View and Configure Logs Without Accessing the File System
The new version now comes with the capability to download server logs, without accessing the file system manually. This is supported by the Management API, CLI, and the dashboard as well. Users can list all the log files available on the server and download any of those as and when needed.
Figure 6: Dashboard view for listing log files
In addition to viewing log files, WSO2 Micro Integrator 1.2 also allows you to view all loggers available, change logging configurations such as changing the new loggers or even add new loggers. These functions are available in the Management API, CLI, and dashboard, and users can use any mechanism based on their preference.
Figure 7: Dashboard view for loggers
Feature 7: Transaction Counter for Services
With WSO2 Micro Integrator 1.2, a user can get to know the number of transactions (requests) that the server is handling on a monthly basis. Once this is enabled, the server will store the aggregated request count in the configured data source both in a raw and encrypted format. The user can query the stored results with the help of the CLI tool in a secure manner. The users have the option to view the total request count for a particular month, or they can generate a report for a time period.
Figure 8: Sample CLI execution for the transaction count
Improvements
This section briefly describes the improvements in the latest release.
Improvement 1: Improvements for the Developer Experience of Micro Integrator Dashboard
WSO2 Micro Integrator Dashboard, (previously known as the monitoring Dashboard) is now enhanced for a better developer experience in terms of viewing server details and managing artifacts/server instances. You can now view data services, server logs/log configurations, and users in addition to the artifact types that you could view earlier. From the managing aspect, you can now activate/deactivate suspended artifacts, enable tracing of artifacts, modify logging configurations, and manage users plugged into an external user store.
Improvement 2: Revamped RabbitMQ Integration
RabbitMQ support for WSO2 Micro Integrator has been completely revamped in order to provide a better and smoother integration experience. All modes of integration (proxy services, inbound endpoints, and message store/processors) have been modified to exploit more of RabbitMQ’s capabilities. Adding modes of rejecting/requesting, better support for DLX, request-response mode of communication come under this.
Improvement 3: Enhanced Experience for Handling Security
WSO2 Micro Integrator 1.2 has much more enhanced security features, in relation to its predecessors. It supports managing sensitive information with its in-built secret repository, with the help of the cipher tool. Users can simply define their secrets in the deployment.toml file and encrypt them and then use them in both server configurations and synapse configurations as needed. WSO2 Integration Studio has also been improved in order to cater to the enhanced security experience, where users can define and encrypt their secrets while building the Docker image for the deployment.
In addition to this, WSO2 Micro Integrator 1.2 also supports external secret management with Docker secrets and Kubernetes secrets. With the help of new encryption capabilities that have been added into the CLI, users can simply get their sensitive information encrypted, and populate them into the containerized environment (either as a Docker secret or a Kubernetes secret). These secrets can be referred from within the synapse configurations by using a wso2:vault-lookup function.
Conclusion
The release of WSO2 Micro Integrator 1.2 is not only cloud-native but also optimized for centralized deployments in order to provide the best experience for its users. Being based on the same synapse integration runtime, it allows a single user to master the deployments of both worlds in a considerably lower time. The improvements that have been made specifically to improve the developer experience add up to these to make their lives even easier.
For more information, please refer to the official product documentation.