Category Archives: Products

What Does WSO2 Identity Cloud Bring To The Table?

One of the things we spoke about at WSO2Con this year was the expansion of our  WSO2 public Cloud offerings. One of those offerings is WSO2 Identity Cloud, which provides the Identity and Access Management (IAM) solution from our well-known WSO2 Identity Server with the ease of use of a cloud service.

Our Initial offering is focused on providing Single Sign-On (SSO) solutions for organizations. Almost all organizations use different applications, either developed in-house or hosted applications like Salesforce and Concur. Having a centralized authentication system with SSO for all the applications increases the efficiency of maintaining systems, centralize monitoring and company security, while also making users’ lives easier.

What are the features offered by WSO2 Identity Cloud?

  • Single Sign-On support with authentication standards – SAML-2.0, OpenID Connect, and WS-Federation.
  • Admin portal provided for organization administrators to log in and configure security for applications. Pre-defined templates of security configurations are available by default for most popular SaaS apps. This list includes Salesforce, Concur, Zuora, GotoMeeting, Netsuite, AWS.
  • On-premise-user-store agent. Organizations can connect local LDAPs with Identity Cloud (without sharing LDAP credentials with Identity Cloud) and let users in the LDAP to access applications with SSO.
  • Identity Gateway.  Act as a simple application proxy that intercepts application requests and applies security checks.
  • User portal. Provides a central location for the users of an organization to log in and discover applications, while applications can be accessed with single sign-on.

Why you should go for a Cloud solution?

If you have following concerns, then a cloud solution is the best fit for you.

  • Facilitating infrastructure – you don’t have to spend money on additional infrastructure with the Cloud solution.
  • System maintenance difficulties – If you do an on-premise deployment, then there should be a dedicated team allocated to ensure the availability of the system and troubleshoot issues; with the Cloud solution, the  WSO2 Cloud team will take care of such things.
  • Timelines – Identity Cloud is tested, stable solution. This will cut down the deployment finalizing and testing times that you should spend on an on-premise deployment.

With all of this comes cost savings, especially because there’s no cost involved for infrastructure or maintenance with the cloud solution.

You can register for WSO2 Identity Cloud and try out for free – http://wso2.com/cloud/ and give us your feedback on bizdev@wso2.com or dev@wso2.org.

Introducing WSO2 Enterprise Integrator 6.0

WSO2 started out as a middleware company. Since then, we’ve realized – and championed the fact that our products enable not just technological infrastructure, but radically change how a company works.

All over the world, enterprises use our products to maximize revenue, create entirely new customer experiences and products, and interact with their employees in radically different ways. We call this digital transformation – the evolution of a company from one age to another, and our role in this has become more a technology partner than a simple software provider.

In this realization, we’ve announced WSO2 Enterprise Integrator (EI) 6.0. Enterprise Integrator brings together all of the products and technologies WSO2’s created for the enterprise integration domain – a single package of digital transformation tools closely connected together for ease of use.

When less is more

Those of you who are familiar with WSO2 products will know that we had more than 20 products across the entire middleware stack.

The rationale behind having such a wide array of products was to enable systems architects and developers to pick and choose the relevant bits that are required to build their solution architecture. These products were categorized into several broad areas such as integration, analytics, Internet of Things (IoT) and so on.

We realized that it was overwhelming for the architects and developers to figure out which products should be chosen. We also realized that digital transformation requires these products to be used in certain common patterns that mirrored five fields: Enterprise Integration, API Management, Internet of Things, Security and Smart Analytics.

In order to make things easier for everyone, we decided to match our offerings to how they’re used best. In Integration, this means we’ve combined the functionality of the WSO2 Enterprise Service Bus, Message Broker, Data Services Server and others; now, rather than including and setting up many many products to implement an enterprise integration solution you can simply download and run Enterprise Integrator 6 (EI 6.0).

What’s it got?

EI 6.0 contains service integration or service bus functionality. It has data integration, service, and app hosting, messaging, business processes, analytic and tooling. It also contains connectors which enable you to connect to external services and systems.



The package contains the following runtimes:

  1. Service Bus

Includes functionality from ESB, WSO2 Data Services Server (DSS) and WSO2 App Server (AS)

  1. Business Processes

Includes functionality of WSO2 Business Process Server (BPS).

  1. Message Broker

Includes the functionality of WSo2 Message Broker (MB). However, this is not to be used for purely message brokering solutions; this runtime is there for guaranteed delivery integration scenarios and Enterprise Integration Patterns (EIPs).

  1. Analytics

The analytics runtime for EI 6.0, useful for tracking performance, tracing mediation flows and more.

In order to provide a unified user experience, we’ve made some changes to the directory structure. This is what it looks like now:

The main runtime is the integrator or service bus runtime and all directories relevant to that runtime are at the top level.

This is very similar to the directory structure we use for other WSO2 products; the main difference is the WSO2 directory, under which the other runtimes are available.

Under the other runtimes, you find the same directory structure as the older releases of those products, as shown below.

One might ask why we’ve included multiple runtimes instead of putting everything in a single runtime. The reason for doing so is the separation of concerns. Short running, stateless integrations will be executed on the service bus runtime while long-running and possibly stateful integrations will be executed on the BPS runtime. We also have optional runtimes such as message broker and analytics which will be required only for certain integration scenarios and when analytics are required, respectively.

By leaving out unnecessary stuff, we can reduce the memory footprint and ensure that only what is required is loaded. In addition, when it comes to configuration files, only files related to a particular runtime will be available under the relevant runtime’s directory.

On the Management Console

There’s also been a change to the port that the management console uses. The 9443 servlet transport port is no longer accessible; we now use the 8243 HTTPS port. Integration services, web apps, data services and the management console are all accessible only on the passthrough transport port, which defaults to 8243.

Tooling

Eclipse-based tooling is available for the main integration and business process runtimes. For data integration, we recommend using the management console itself from the main integration runtime.


Why 6.0?

As the name implies, EI is an integration product. The most widely used product in the integration domain is the WSO2 Enterprise Service Bus (ESB), which in the industry is known to run billions of transactions per day. EI is in effect the evolution of WSO2 ESB 5.0, adding features coming from other products. Thus, it’s natural to dub this product 6.0 – the heart of it is still the same.

However, we’ve ensured that the user experience is largely similar to what it was in terms of the features of the previous generation of products.  The Carbon platform that underlies all of our products made it easy to achieve that goal.

Migration to EI 6.0

The migration cost from the older ESB, BPS, DSS and other related products to EI 6.0 is minimal. The same Synapse and Data Services languages, specifications and standards have been followed in EI 6.0. Minimal changes would be required for deploying automation scripts such as Puppet scripts -the directory structures are still very similar, and the configuration files haven’t changed.

Up Next: Enterprise Integrator 7.0

EI 6.0 is based on several languages – Synapse for mediation, BPMN and BPEL for business processes, DSS language for data integration.

A user who wants to implement an integration scenario involving mediation, business processes, and data integration has to learn several languages with different tooling. While it’s effective, we believe we can do better.

At WSO2Con 2017, we just unveiled Ballerina, an entirely new language for integration. EI 7.0 will be completely based on Ballerina – a single language and tooling experience. Now the integration developer can concentrate on the scenario, and implement it using a single language and tool with first level support for visual tooling using a sequence diagram paradigm to define integration scenarios.

However, 7.0 will come with a high migration cost. Customers who are already using WSO2 products in the integration domain can transition over to EI 6.0 – which we’ll be fully supporting – while planning on their 7.0 migration effort in the long term; the team will be working on tooling which will allow migration of major code to Ballerina.

WSO2 will continue to develop EI 6 and EI 7 in parallel. This means new features and fixes will be released as WUM updates and newer releases of the EI 6.0 family will be available over the next few years so that existing users are not forced to migrate to EI 7.0. This is analogous to how Tomcat continues to release 5.x, 6.x, 7.x and so on.


EI 6.0 is available for download at wso2.com/integration and on github.com/wso2/product-ei/releases. Try it out and let us know what you think – it’s entirely open source, so you can take a look under the hood if that takes your fancy. To report issues and make suggestions, head over to https://github.com/wso2/product-ei/issues.

Need more information? Looking to deploy WSO2 in an enterprise production environment? Contact us and we’ll get in touch with you.

 

What is new in WSO2 Identity Server 5.3.0?

Since its launch in 2007, WSO2 Identity Server (WSO2 IS) has become an industry leading product in the open source, on-premise IAM space. It’s trusted by both the government and private sectors for large scale deployments ranging up to millions of users.

Apart from the open standard support, WSO2 IS has a solid architecture to build a strong identity ecosystem around it. More than 40 connectors are now available for you to download from WSO2 Connector Store – including SMS OTP, Email OTP, TOTP (Google Authenticator), Duo Security, mePIN, RSA, FIDO U2F  – and many more. All these connectors are released under the same open source Apache 2.0 license, as of the product.

The focus of WSO2 Identity Server 5.3.0 is to build and enhance features around Identity/Account Administration and Access Governance. Here are the new features introduced in WSO2 Identity Server 5.3.0:

  • Identify and suspend user accounts that have been idle for a pre-configured amount of time. Prior to account suspension, the administrator can set up the notification system to notify the user with a warning that the account will be suspended.
    For instance, if a user has not logged in to his/her account for 90 days, the user will be notified that his account will be suspended within the next 7 days if there continues to be no activity, after which the account will be suspended.
  • A new REST API was introduced to recover a lost/forgotten password, i.e., by using email notifications or secret questions. It is also possible to recover the username if forgotten. This extends the functionality of the SOAP service WSO2 IS had before 5.3.0.
  • The administrator can trigger the password reset for a given user. This may be required if the user forgets the credentials and then makes a request to the administration to reset the password — and also in cases where the credentials get exposed to outsiders then the administrator can lock the account and enforce password reset.
  • Support for Google reCAPTCHA as a way of brute-force mitigation. The administrator can configure Google reCAPTCHA in the login, password/account recovery and sign up flows.
  • Maintain the history of the user’s passwords according to a pre-configured count. This prevents a user from using a password he/she has used in the recent past. For example, if you configure a count of 5, the user will be prevented from reusing his/her last 5 passwords as the current password.
  • The administrator can monitor all the login sessions — and can selectively terminate.
  • Enforce policies to control outbound user provisioning operations. For example, you can provision users having the salesteam role to Salesforce and anyone having an email address with the domain name foo.com to Google Apps.
  • Partition users by service providers. WSO2 IS had support for multiple user stores since its version 4.5.0. With this new feature, the administrator can specify against which user store the user should authenticate, by the service provider. For example, only the users in the foo user store will be able to access the foo service provider.
  • Enforce policies during the authentication flow. The administrator can, for example, enforce a policy which states only the users having the salesteam role can access Salesforce, and only during a weekday from 8 AM to 4 PM.
  • Improvements for the JIT provisioning flow. The administrator can now specify mandatory attribute requirements for JIT provisioning and if any of those are missing, WSO2 IS will prompt the user to enter the values for the missing attributes.
  • Improvements for identity analytics. With WSO2 IS 5.3.0 the identity administrator can get alerts for abnormal and suspicious login sessions.

In addition to the above set of features, WSO2 IS 5.3.0 also introduced a set of enhancements for its existing open standards.

  • SAML 2.0 Metadata Profile
  • SAML 2.0 Assertion Query/Request Profile
  • OpenID Connect Dynamic Client Registration
  • OAuth 2.0 Token Introspection
  • OpenID Connect Discovery
  • JSON/REST profile of XACML

WSO2 IS 5.3.0 is now the best it’s ever been. We hope you will find it quite useful to address your enterprise identity management requirements, and we’re more than happy to hear your feedback/suggestions — please feel free to post them to bizdev@wso2.com or dev@wso2.org.

Implementing an Effective Deployment Process for WSO2 Middleware


Image reference: https://www.pexels.com/photo/aerospace-engineering-exploration-launch-34521/

At WSO2, we provide middleware solutions for Integration, API Management, Identity Management, IoT and Analytics. Running our products on a local machine is quite straightforward: one just needs to install Java, download the required WSO2 distribution, extract the zip file and run the executable.

This provides a middleware testbed for the user in no time. If the solution needs multiple WSO2 products, those can be run on the same machine by changing the port-offsets and configuring the integrations accordingly.

This works very well for trying out product features and implementing quick PoCs. However, once the preliminary implementation of the project is done, a proper deployment process is needed for moving the system to production. 

Any software project needs at least three environments for managing development, testing, and the live deployments. More importantly, a software governance model would be needed for delivering new features, improvement, bug fixes and managing the overall development process.

This becomes crucial when the project implements the system on top of a middleware solution. Both middleware and application changes will need to be delivered. There might be considerable amounts of prerequisites, artifacts and configurations. Without having a well-defined process, it would be difficult to manage such projects efficiently.

A High-Level Examination

One would have to consider the following points would need to be considered when implementing an effective deployment process:

  • Infrastructure

WSO2 middleware can be deployed on physical machines, virtual machines and on containers. Up to now most deployments have been done on virtual machines.

Around 2015, WSO2 users started moving towards container-based deployments using Docker, Kubernetes and Mesos DC/OS. As containers do not need a dedicated operating system instance, this cuts down resource requirements for running an application – in contrast to a VM. In addition, the container ecosystem makes the deployment process much easier using lightweight container images and container image registries.

We provide Puppet Modules, Dockerfiles, Docker Compose, Kubernetes and Mesos (DC/OS) artifacts for automating such deployments.

  • Configuration Management

The configuration for any WSO2 product can be found inside the relevant repository/conf folder. This folder contains a collection of configuration files corresponding to the features that the product provides.

The simplest solution is to maintain these files in a version control system (VCS) such as Git. If the deployment has multiple environments and a collection of products, it might be better to consider using a configuration management system such as Ansible, Puppet, Chef or Salt Stack for reducing configuration value duplication.

We ship Puppet modules for all WSO2 products for this purpose.

  • Extension Management

WSO2 middleware provides extension points in all WSO2 products for plugging in required features.

For example, in WSO2 Identity Server a custom user store manager can be implemented for connecting to external user stores. In the WSO2 Integration products, handlers or class mediators can be implemented for executing custom mediation logic.  Almost all of these extensions are written in Java and deployed as JAR files. These files will simply need to be copied to the repository/components/lib folder or the repository/components/dropins folder if they are OSGi compliant.

  • Deployable Artifact Management

Artifacts that can be deployed in repository/deployment/server folder fall under this category. For, example, in the ESB, proxy services, REST APIs, inbound endpoints, sequences, security policies can be deployed in runtime via the above folder.

We recommend that you create these artifacts in WSO2 Developer Studio (DevStudio) and package them into Carbon Archive (CAR) files for deploying them as collections. WSO2 DevStudio provides a collection of project templates for managing deployable files of all WSO2 products. These files can be effectively maintained using a VCS.

These files can be effectively maintained using a Version Control System.

  • Applying Patches/Updates

Patches were applied to a WSO2 product by copying the patch<number> folder which is found inside the patch zip file to the repository/deployment/patches/ folder.

We recently introduced a new way of applying patches for WSO2 products with WSO2 Update Manager (WUM). The main difference of updates, in contrast to the previous patch model, is that fixes/improvements cannot be applied selectively; it applies all the fixes issued up to a given point using a CLI. This is the main intention of this approach.

  • Lifecycle Management

In any software project it is important to have at least three environments – one for managing development, one for testing and one for production deployments.  New features, bug fixes or improvements need to be first done in the development environment and then moved to the testing environment for verification. Once the functionality and performance are verified the changes can be applied in production (as explained in the “Rolling Out Changes”) section.

The performance verification step might need to have resources identical to the production environment for executing load tests. This is vital for deployments where performance is critical.

With our products, changes can be moved from one environment to the other as a delivery.  Deliveries can be numbered and managed via tags in Git.

The key advantage of using this approach is the ability to track, apply and roll back updates at any given time.

  • Rolling Out Changes

Changes to the existing solution can be rolled out in two main methods:

1. Incremental Deployment (also known as Canary Release).

The idea of this approach is to incrementally apply changes to the existing solution without having to completely switch the entire deployment to the new solution version. This gives the ability to verify the delivery in the production environment using a small portion of the users before propagating it to everyone.

2. Blue-Green Deployment

In Blue-Green deployment method, the deployment is switched to the newer version of the solution at once. It would need an identical set of resources for running the newer version of the solution in parallel to the existing deployment until the newer version is verified. In case of failure, the system can be switched back to the previous version via the router. Taking such approach might need a far more thorough testing procedure compared to the first approach.

Deployment Process Approach 1

This illustrates the simplest form of executing a WSO2 deployment effectively.

In this model the configuration files, deployable artifacts and extension source code are maintained in a version control system. WSO2 product distributions are maintained separately in a file server. Patches/updates are directly applied to the product distributions and new distributions are created. The separation of distributions and artifacts allows product distributions to be updated without losing any project content.

As shown by the green box in the middle, a deployable product distribution is created, combining the latest product distributions, configuration files, deployable artifacts and extensions. Deployable distributions can be extracted on physical, virtual machines or containers and run. Depending on the selected deployment pattern, multiple deployable distributions will need to be created for a product.

In a containerized deployment, each deployable product distribution will have a container image. Depending on the containerized platform, a set of orchestration and load balancing artifacts might also be used.

Deployment Process Approach 2

In the second approach, a configuration management system has been used for reducing the duplication of the configuration data and automating the installation process.

Similar to the first approach, deployable artifacts, configuration data and extension source code are managed in a version control system. Configuration data needs to be stored in a format that is supported by the configuration management system.

For an example, in a Puppet configuration, data is either stored in manifest files or Hiera YAML files. Deployable WSO2 product distributions are not created. Rather, that process is executed by the configuration management system inside a physical machine, virtual machine or in a container at the container build time.

In conclusion

Any of the deployment approaches we’ve spoken about above can be followed with any infrastructure. If a configuration management system is used, it can be used for installing and configuring the solution on virtual machines and as well as on containers. The main difference with containers is that configuration management agent will only be triggered at the container image build time. It may not be run in the when the container is running.

If a configuration management system is used, it can be used for installing and configuring the solution on virtual machines and as well as on containers. The main difference with containers is that configuration management agent will only be triggered at the container image build time. It may not be run in the when the container is running.

At the end of the day, a proper deployment process is essential. For more information and support, please reach out to us. We’d be happy to help.

Meet WSO2 EMM 2.2.0!

We’re excited to announce yet another landmark of our EMM story:  the latest version WSO2 EMM 2.2.0! WSO2 EMM comes with a host of device management, app management and analytics features that benefit IT admins as well as device owners themselves.

Let’s explore some of the new key features of this release.

Device Management

The latest release comes with improved APIs for better extensibility, advanced WiFi profiles and supports device restrictions available in Android 5.0 – Lollipop upwards.

Advanced WiFi Profiles

Some organizations prefer to configure enrolled devices over-the-air (OTA). The previous WSO2 EMM version supported only WEP (simple profile with only SSID and password input) and with 2.2.0 organizations will be able to configure enrolled devices with advanced WiFi profile types, such as EAP, WPA2 and enabling TLS/TTLS.

Device Restrictions

WSO2 EMM 2.2.0 supports all device restrictions (e.g. network configuration, VPN configuration, volume control) available from Android 5.0 – Lollipop upwards. For the complete list of supported devices restrictions, refer to our official documentation (Note: camera setting was delivered in a previous release).

App Catalog at Your Service

In the previous WSO2 EMM distribution, when a mobile application needs to be installed on a device either the admin will have to push applications to the mobile device via the WSO2 EMM Management Console or the device owner will have to be granted access to the Management Console, which is not a practical scenario.

With 2.2.0, WSO2 EMM will have a standalone mobile app called ‘App Catalog’. The App Catalog lists all mobile apps the device owner is permitted to install. Device owners will be facilitated to install mobile apps with just a click of a button and to uninstall and remove them as well.

Whitelisting and Blacklisting Apps

With this feature admins will able to whitelist and blacklist mobile apps already installed in the App Store, so that a specific set of mobile applications are provisioned to device owners. This will also enable fencing unknown malicious mobile apps from accessing corporate data.

Room to Grow – Let’s OEM

With this release WSO2 EMM unlocks a host of features capable of underpinning OEM efforts for organizations using custom Android devices as part of their business strategy (e.g. medical devices, point-of-sale devices, kiosks). Managing custom devices is two-fold; you can either maintain custom firmware or use custom apps signed by the device vendor (or by the firmware key provided by the device vendor). The 2.2.0 distribution comes with a system service app that can be installed on the device and thereby used to perform root operations on the device.

emm 2.2

Automatic Device Enrollment

With this, admins will be able initiate the device auto-enrollment by entering serial numbers via the Management Console for the required devices. Once corresponding devices are handed over to device owners, device owners will be facilitated to select the relevant serial number from the device and generate a one-time-token (OOT), which expires within a predefined duration. To complete the enrollment, you can either type in the OOT or simply scan the QR code.

This will increase the speed of enrolling a large number of devices with a few steps with less device user intervention.

Over-The-Air Firmware Upgrade

This feature will allow admins to upgrade device firmware (apps written to device ROM) via the WSO2 EMM Management Console to one/more devices in one go (e.g. a firmware upgrade to all COPE devices). Device owners, on the other hand, need not worry about manually obtaining the latest firmware, as upgrades will be auto-installed.

Silent App Installation, Update, and Removal

In the previous WSO2 EMM version, app installations would only take place subsequent to a user confirmation. With 2.2.0, apps can be installed, updated, or even removed from the device without the device owner’s consent.

Device Hard Lock

This enables admins to completely block a device user that can only be revoked by an admin. This will help organizations to screen out device users who breach organizational policies.

Device Reboot

This facilitates admins to remotely reboot Android devices via the Management Console.

How are my Devices Doing?

WSO2 EMM 2.2.0 offers an array of features to keep you up-to-date around your device portfolio.

Analytics Dashboard

The WSO2 EMM Device Monitoring Dashboard provides admins with insights into unmanaged and non-compliant devices, device distribution by platform, and BYOD/COPE ownership and connectivity.

Device Details

Admins can view both dynamic and static device related information via the WSO2 EMM Management Console. Viewable static data include memory, CPU details, and OS version. Viewable dynamic data include CPU/memory utilization, battery level, installed apps, connectivity strength, power status (i.e. on battery or plugged into a power source), and GPS location.

Alerts on Alerts

The previous WSO2 EMM Management Console facilitated admins to send alerts to Android devices; from WSO2 EMM 2.2 onwards, admins will be notified on the alert delivery and the device owner’s response to alerts as well, i.e. be notified on whether the alert was delivered, displayed, or dismissed. In addition, admins will be able to send custom alert types as well.

WSO2 Enterprise Mobility Manager (WSO2 EMM) is a 100% open source comprehensive platform supporting iOS, Android and Windows devices, which help organizations to deal with both corporate-owned, personally-enabled (COPE) devices and employee-owned devices with the bring your own device (BYOD) program.

You can download the product here and try it out for yourself. If you come across any issues please feel free to report them via the public JIRA.

WSO2 API Manager for Small and Medium Enterprises

WSO2 API Manager provides a very light-weight, robust and a scalable API management solution for organizations who want to manage APIs. WSO2 API Manager comes as a single product distribution but has five different components and can be deployed as individual components allowing greater flexibility in deployment while adhering to internal security policies of the organization. Running WSO2 API Manager as components allows more flexibility when scaling the solution and optimizes the resource usage.

WSO2 API Manager is deployed as a distributed deployment at many large organizations that expose APIs for both internal and external service consumers with millions of API calls serviced during a day. These organizations rely on WSO2 API Manager to deliver critical services to its stakeholders, which has a direct impact on their business operations. A distributed deployment can typically cater over 2500 API calls a second which is a staggering 200 million+ API calls a day.

Even though this kind of a requirement holds true for large-scale organizations, there are many organizations who need to start small and want a simple API manager deployment to support their requirements. WSO2 provides two solutions for such customers.

  1. On-Cloud – WSO2 API Cloud, which is a subscription-based API manager solution that you can subscribe to and pay based on the API usage. WSO2 API cloud offer plans ranging from 250, 000 to 50 million API calls a day.
  2. On-Premise – WSO2 offers an all-in-one API manager deployment pattern that allows you to deploy the WSO2 API Manager on-premise as a single instance. This type of on-premise deployment can support up to 500 API calls a second (nearly 43 million API calls a day). More details on this option are given below.

An all-in-one API Manager deployment has i5 components (Gateway, Key Manager, Store, Publisher and Traffic Manager) deployed inside a single product distribution. The deployment is very easy to setup and WSO2 provides a very comprehensive setup guide1 that provide step by step instructions on how to deploy the product. All-in-One API Manager can be deployed as a single active node, in Active/Passive mode or as an Active/Active node cluster. The diagrams shown below visually illustrates the 3 types of possible deployment patterns.

api-manager-for-smes-1

Fig 1: Single API Manager node

api-manager-for-smes-2

Fig 2: Active/Passive deployment

api-manager-for-smes-3

Fig 3: Active/Active API Manager deployment

It is recommended to deploy the API Manager fronted by a load balancer for all three deployment patterns. If APIs need to be exposed for external consumption it is recommended to use a Reverse Proxy or a Firewall along with the load balancer. A Reverse Proxy or a Firewall is required to route legitimate traffic from outside the network to the API Manager node that resides in the LAN. It is recommended to configure the API Manager to work with an external RDBMS to ensure reliability. If API Manager is deployed as an active/active or active/passive setup it would require a content synchronization mechanism such as Rsync2 (DeltaCopy or cwRsync for Windows) to synchronize APIs between the two nodes.

With the new all-in-one API Manager deployment pattern, there are multiple options for an organization to structure their deployment. If you are building a large-scale API Manager deployment then it would be recommended to start with a distributed API Manager deployment. However, if you want to start small and make the API Management platform available quickly this type of a deployment would be the ideal choice.

WSO2 API Manager 2.0.0: Better Statistics

Introduction

Any API management tool requires the ability to view statistics – accurately and with details essential to the operation of APIs. WSO2 API Manager 2.0.0 introduces powerful analytics features along these lines by using the capabilities of WSO2 Data Analytics Server (WSO2 DAS), bringing all API, application and subscription related statistics to one location.

image05

New additions

In order to help you make sense of these analytics, we’ve added the following graphs:

1.Published APIs Over Time

This graph can be used to check API creation rates. A user can check APIs based on the API provider or can view statistics related to all API providers.

image05

2. API Latency

The API Latency breakdown graph can be used to check time consumption for each level in the message flow. It shows total time taken to handle a request and provides the time each component spends on that request – for example, a user can check the amount of time spent on the authentication process and so on. This graph is useful for identifying delays in the API.

image013. API Usage Across Geo Locations

This graph provides statistics based on the location of the API request. It uses X-Forwarded-For header to identify the location of the user.

image06

4. API Usage Across Usage Agent

This graph provides information related to the access mechanism of the API, and helps provide insight into the kind of devices and platforms used to access the API – mobiles, computers and so on.

image08

We’ve also added an Applications Created Over Time graph under the Applications section. As with the API creation graph, this provides statistics related to application creation. Users can drill down the data based on the registered user and the API in question.

image02

In addition to these come the subscription graphs: Developer Signups Over Time and Subscriptions Created Over Time, both of  which are self-explanatory.

image00

image07

How These Statistics Work

WSO2 Analytics Server for API Manager is essentially a fully functional WSO2 DAS server.

image03

In a nutshell, WSO2 API Manager 2.0.0 sends the following event streams to API Manager Analytics:

org.wso2.apimgt.statistics.request:1.1.0

org.wso2.apimgt.statistics.response:1.1.0

org.wso2.apimgt.statistics.fault:1.0.0

org.wso2.apimgt.statistics.throttle:1.0.0

org.wso2.apimgt.statistics.workflow:1.0.0

org.wso2.apimgt.statistics.execution.time:1.0.0

org.wso2.analytics.apim.alertStakeholderInfo:1.0.0
The APIM Analytics Server process these events and writes the summarized data to a shared database. WSO2 API Manager then queries statistics from this database to display in the API Manager Publisher and Store.

Footnotes

One significant change in WSO2 API Manager 2.0.0 statistics is the removal of the graphical user interface (GUI) for configuring statistics. In WSO2 API Manager 1.10, a GUI was provided to handle this DAS-related configuration. We’ve moved this functionality back into the api-manager.xml file.

We’ve also taken out the DAS Rest client – with 2.0.0, the default and only implementation is APIUsageStatisticsRdbmsClientImpl, which uses an RDBMS client to get data from the database.

Since this is essentially a DAS server, you can use features such as gadgets and realtime analytics (using Siddhi) for better or more complex visualizations of the events sent from WSO2 API Manager.

Keep your WSO2 products up to date with WUM

We at WSO2 continuously improve our products with bug fixes, security fixes , and various other improvements. Every major release of our WSO2 products is followed by a series of periodic dot releases that roll up all these recent updates.

We want everyone evaluating, developing on, or preparing WSO2 products for production deployments to have the best experience and especially not trip over a bug that has already been fixed! WSO2 Update Manager (WUM) makes this process much easier – now you can get access to latest updates as and when we release them.

What is WUM?

WUM is a command-line application which allows you to keep WSO2 products up-to-date. It determines which updates are new and relevant for your WSO2 products, downloads and installs them. WUM builds you a new product distribution with all the relevant updates; this distribution is ready to deploy into your development or production environment. Note that WUM will not push updates into production: you have full control over when and how you want to such pushes to occur.

Updates are available for the following products and versions. We are in the process of adding more products and versions.

  • WSO2 Enterprise Service Bus 5.0.0
  • WSO2 Enterprise Service Bus 4.9.0
  • WSO2 Enterprise Service Bus Analytics 5.0.0
  • WSO2 API Manager 2.0.0
  • WSO2 API Manager Analytics 2.0.0

Note that fresh WSO2 product distributions downloaded via the wso2.com site or using WUM, are licensed under Apache 2. However, product distributions built by WUM with updates installed are licensed under WSO2 Update License 1.0. The updated product distributions are not licensed for use in production without a valid WSO2 subscription. Visit http://wso2.com/support for more information about WSO2 subscription.

Installing WUM

You can download WUM from http://wso2.com/update. Download the binary release suitable for your operation system and processor architecture. Binary distributions are available for Linux, Mac OS, and Windows (follow installation instructions available on the same download page).

Once the installation is completed, open a new terminal and execute the following command. You should get the help page with all the available commands.

$ wum

Initialize WUM with your WSO2 credentials

You need to have a WSO2 account to get started with WUM. If you don’t have an account with WSO2, please create one here.

$ wum init
You need a WSO2 account to start using wum.
Don't have one yet? Sign up at https://wso2.com/user/register

Please enter your WSO2 credentials to continue
Username: sameera@wso2.com
Password for 'sameera@wso2.com': 
Authenticating...
Done!

-- Welcome to WUM 1.0-beta --

* Find more information at http://wso2.com/update
* Documentation: https://docs.wso2.com/display/ADMIN44x/Updating+WSO2+Products
* JIRA: https://wso2.org/jira/browse/WUM

What's next? Have a look at the following list of wum commands:

Add WSO2 products to your product repository
  wum search  Search WSO2 products         
  wum add     Add or download WSO2 products

Update WSO2 products available in your product repository
  wum check-update Check for new updates    
  wum update       Update your WSO2 products

Manage WSO2 products available in your product repository
  wum list      List WSO2 products           
  wum describe  Show details of WSO2 products
  wum delete    Delete WSO2 products

Use "wum [command] --help" for more information about a command.

Use "wum [command] --help" for more information about a command.

Local product repository

WUM maintains a local product repository in your machine where it stores WS02 product distributions. Whenever you update a product, WUM creates a new distribution with all these updates and stores it in this repository. By default, WUM creates this repository in the user’s home directory, but you have the option to change this as well.


Search and add WSO2 products

Before you update any WSO2 product, you need to add the product to your local product repository. You can either download a product or add an already downloaded product. If you are not sure about what product name to use, you can use the ‘wum search’ command. If you run the search command without any arguments, it will list all the WUM supported WSO2 products.

$ wum search
Connecting to WSO2 Update...

wso2esb-5.0.0 - Enterprise Service Bus
wso2esb-analytics-5.0.0 - Enterprise Service Bus Analytics
wso2am-2.0.0 - API Manager
wso2am-analytics-2.0.0 - API Manager Analytics
wso2esb-4.9.0 - Enterprise Service Bus

What's next?
  use "wum add <product>" to download a product
  e.g "wum add wso2esb-5.0.0" to download Enterprise Service Bus 5.0.0

You can also search using keywords.

$ wum search api manager
Connecting to WSO2 Update...

wso2am-2.0.0 - API Manager
wso2am-analytics-2.0.0 - API Manager Analytics

What's next?
  use "wum add <product>" to download a product
  e.g "wum add wso2am-2.0.0" to download API Manager 2.0.0

If you look at the examples given in the “what’s next” section, you would see the expected format of the product name to use. Use the following command if you want to add the latest version of wso2esb.

$ wum add wso2esb
Connecting to WSO2 Update...
The following product(s) will be downloaded.
  wso2esb-5.0.0.zip
After this operation, 218.1MB of additional disk space will be used.
Do you want to continue? [Y/n]
Downloading product wso2esb-5.0.0... [2.4MB/218.1MB]

If you want to add a specific version of wso2esb then you need to specify the version as follows:

$ wum add wso2esb-4.9.0

You also have the option to add an already downloaded product distribution using the add command.

$ wum add --file ~/user/wso2/dist/wso2esb-5.0.0.zip

Now try the following command to see the list of add products.

$ wum list

Update WSO2 products

As I’ve explained earlier, WUM creates a new product zip file with all the updates installed. But if you want to check whether updates are available before you attempt to update, you can use the “check-update” command as follows.

$ wum check-update wso2esb-4.9.0
Connecting to WSO2 Update...

Checking for latest updates for wso2esb-4.9.0...
96 updates are available
[WARNING] 11 critical security updates. WSO2 strongly recommends that you install these updates now.

What's next?
  use "wum update" to install latest updates

Now let’s update the products using the “update” command

$ wum update wso2esb-4.9.0
Connecting to WSO2 Update...

Validating your subscription status for product wso2esb...
[WARNING] Your credentials are not associated with an active WSO2 subscription.
Please remember that updates are not licensed for use in production without a valid WSO2 subscription.
See: http://wso2.com/update for more details.

Updating wso2esb-4.9.0...
96 updates are available
Downloading updates...
Installing updates...
Preparing update summary...
Building updated product...
Update summary:
  Installed updates: 96
  * [WARNING] WSO2 strongly recommends to use this updated product for production as soon as possible.
  Security updates: 11
  Updated Product: /Users/sameera/.wum-wso2/products/wso2esb/4.9.0/wso2esb-4.9.0.1472889183206.zip
  * More information about updates are available inside the above product archive
  Update summary(pdf): (product-archive)/updates/summary-2016-09-08T19-23-50/update-summary-1472889183206.pdf
  * Configuration changes are available
  Change log: (product-archive)/updates/summary-2016-09-08T19-23-50/change-log-1472889183206.txt

What's next?
  use "wum list [<product-pattern>]" to list products in your local repository
  use "wum describe [<product-pattern>]" to get more details of products

If you don’t have an active WSO2 subscription for wso2esb, you would get a warning as above. This means you can use the updated product for your development activities, but you are not supposed to use it in production.

Once the update is completed, you should be able to see the updated product files in your local repository. We version updated product distributions using the timestamp of the latest update.

$ wum list
Product        Updated              Filename                       
wso2am-2.0.0   08 Sep 16 18:01 IST  wso2am-2.0.0.1473322072976.zip 
wso2am-2.0.0   -                    wso2am-2.0.0.zip               
wso2esb-4.9.0  08 Sep 16 19:24 IST  wso2esb-4.9.0.1472889183206.zip
wso2esb-4.9.0  -                    wso2esb-4.9.0.zip

Manage your product repository

There are several WUM commands available for you to manage your local product repository. The “wum list” command lists all the available products. If you want to get more information about a product, you can use the “wum describe” command. It gives you the location of the product file and update history, among other information.

$ wum describe wso2esb-4.9.0.1472889183206.zip
Filename:  wso2esb-4.9.0.1472889183206.zip
Product Name:  wso2esb
Product Version: 4.9.0
Kernel Version:  4.4.1
Last Updated Time: 2016-09-08T19:23:55+05:30
Product File Path: /Users/sameera/.wum-wso2/products/wso2esb/4.9.0/wso2esb-4.9.0.1472889183206.zip
No Of Times Updated: 1
Installed Updates: 96
Update History:
 Date                       Installed Updates  Username        
 2016-09-08T19:23:55+05:30  96                 sameera@wso2.com

In case you want clean up your product repository by deleting files, you can use the “wum delete” command.

$ wum delete wso2esb-4.9.0
The following product file(s) will be deleted.
  wso2esb-4.9.0.1472889183206.zip
  wso2esb-4.9.0.zip
Do you want to continue? [y/N]

The above command will delete all the wso2esb-4.9.0 product files including the updated files. List, describe and delete commands accepts a product pattern; therefore you can filter out products by given a particular pattern.


We’ve released the Beta version of WUM. Try it out and let us know what you think.

Meet the WSO2 Enterprise Service Bus 5.0

The WSO2 Enterprise Service Bus is renowned as a lightweight, 100% open source ESB; it helps everything from big, digital-first companies (like eBay) to transnational governing bodies (like the Eurasian Union) achieve the kind of enterprise integration scenarios they need to work. We’ve now released version 5.0 of the WSO2 ESB – and here’s why you should take a look. 

As part of our new product strategy, the WSO2 ESB 5.0 release is composed of three key components – the runtime, tooling, and analytics. We’ve also added several new, major features:

  • Mediation Debugger for error detection in mediation flows
  • Data mapper to convert and transform data
  • Mediation statistics and tracing using ESB Analytics component
  • Support for JMS 2.0 and WebSocket transports

Interested? Let’s explore in more detail.

WSO2 ESB Tooling 5.0

The new WSO2 Developer Studio-based ESB tooling plug-in is built to work on top of Eclipse Mars2, the widely used IDE. It offers developers with improved usability and brings some new capabilities to the table:

Mediation debugging

At the operational level, a series of mediators called sequences determine the behavior of messages. Users were previously unable to figure out if these mediators -including their inputs, processing, and outputs – were operating as intended. The newly introduced Mediation Debugger allows users to debug this message mediation flow proactively, to identify any errors and rectify them in the development environment itself (figure 1).

image05Figure 1: Debugging mediation flow

Data conversion and transformation

 

WSO2 Data Mapper is one of the most awaited features by our customers. Having been built as a mediator that can be included in the mediation flow, it is capable of converting the data format of a message between JSON, XML or CSV (figure 2). It also comes with an operations palette that supports a number of operations to help users perform different functions in data mapping (figure 3). For instance, arithmetic functions enable users to add, subtract, multiply, divide or carry out other functions on data as a part of its transformation. Users can easily drag and drop operators into the mapping diagram and edit on the go.

The main categories of operations are listed below.

  • Common
  • Arithmetic
  • Conditional
  • Boolean
  • Type conversion
  • String

We’ve made it more appealing to developers by introducing a graphical mapping experience that also generates the execution files required to run on the Data Mapper Engine.

image09Figure 2: Data mapping

What’s special about WSO2 Data Mapper is that it can function independently of other WSO2 products or components.

image00Figure 3: Adding operations in data transformation

WSO2 ESB Analytics 5.0

The analytics component is a new addition to the ESB that lets users publish information on ESB processes. Previously, users had the ability to connect WSO2 DAS and define their own ESB dashboards. However, we’ve made the experience better by providing an analytics dashboard out of the box, with pre-defined statistics for ESB artifacts such as proxy services, APIs, sequences, endpoints and inbound endpoints. Users have the option here to choose which artifacts’ statistics and/or tracing should be published by enabling or disabling this through the runtime instance.

The dashboard displays a summarized view of the requests processed, their success/ failure rates, throughput, and message count over a specified time period, and top artifacts by request count (Figures 4, 4a, 4b, 4c respectively). For example, users can identify the top sequences, endpoints or the top APIs by request count in separate gadgets.

image04Figure 4: Statistics dashboard

image03Figure 4a: Overall transactions per second graph

image01

Figure 4b: Message count graph

image08

Figure 4c: Top endpoints by request count graph

The dashboard provides many features – from configurable timelines and detailed stats by artifact type to visual additions, such as theming and the customizability to portray your brand and explain statistics (figure 5).

image07

Figure 5: Dashboard with configurable timelines, theming buttons and navigation pane

Tracing mediation flows

Not only is it important to monitor statistics, but the ability to drill down into anomalies encountered in ESB processes is essential for DevOps teams, especially in production environments. Hence, we’ve introduced message tracing capabilities to provide more visibility into message mediation flows so developers can identify problems in live environments.

For a particular artifact type such as an API or proxy service, operational teams can dive into obtaining overall message count, latency and processing details of each message. Consider a scenario where messages pass through a REST API: the message gadget (figure 6) helps DevOps teams proactively identify which messages failed, as shown below.

image06Figure 6: Messages and their status

We’ve also made it possible to further scrutinize into the message flow (figure 7), and dive into a mediator and view its properties in relation to a payload, transport properties and context properties (figure 8), both before and after the mediator. It provides a mechanism by which operational teams can verify if message flows operate as they are intended, and fix any errors.

image02Figure 7: Message flow detailed view

image10Figure 8: Mediator properties

These features also are useful in testing environments to evaluate changes done to artifacts (sequences, APIs, endpoints etc ) or message mediation flows, as a feedback mechanism prior to moving into production.

Support for JMS 2.0 and Websocket

Last but not least, we’ve also added two transports – JMS 2.0 and WebSocket – to be able to support more integration use cases.

With JMS 2.0, the WSO2 ESB supports new messaging features – shared topic subscription, getting the JMSX delivery count and specifying message delivery delays. A typical scenario would be using the JMS endpoint to act as a shared topic listener so that it can connect to shared topic subscribers and help share the workload. We’ve also introduced WebSocket as a transport and inbound protocol to support bidirectional message mediation mostly advantageous in near-real-time communication for IoT, web, and mobile scenarios.

We’ve bumped the product quite a bit in this release and we’d love to hear what you think about the all new WSO2 ESB 5.0.

Go ahead and try it yourself – it’s free to download and use!

What’s new in MSF4J 2.0?

The MSF4J team is excited to announce the latest release of WSO2 Microservices Framework for Java (MSF4J) 2.0!

MSF4J is essentially a lightweight, high performance framework for developing and running microservices.

This release comes with a number of major enhancements and features:

  • An improved thread model and transport architecture
  • Spring and Swagger support
  • ExceptionMapper and StreamingOutput support
  • More powerful form submission commands
  • Circuit Breaker state management based on Nygard’s pattern

Let’s examine these in more detail.

Improved thread model and transport architecture

The default thread model in MSF4J 1.0 included the Netty boss threads handing over the requests to Netty worker threads, and the microservice requests being executed using the Netty worker threads. However, this leads to the problem of the Netty worker thread pool getting exhausted if the threads took long to execute or were blocked. That results in requests being dropped.

With the new thread model, service requests are executed using a separate executor thread pool (application thread pool) by default. There is also the option of handing over the incoming request to an LMAX disruptor which is useful in certain scenarios. As a user, you will have full control over configuring these thread pools.

image05

MSF4J thread model. S in the diagram is your MSF4J service.

MSF4J 1.0 also had a direct dependency on Netty, but with 2.0, we’ve brought in an abstraction called CarbonMessage. CarbonMessage will represent the request and response messages, irrespective of the underlying transport technology.

Spring support

image01

Spring is a popular framework among developers because it provides a very simple dependency injection (DI) mechanism which makes life easy for developers. We decided to support Spring owing to popular user demand for dependency injection support.

MSF4J now supports a Spring-native programming model. You can now write your MSF4J microservices, Interceptors, ExceptionMappers and configuration as Spring beans and wire them up at runtime.

The standard Spring configuration annotations such as @Component, @AutoWired and other Spring features work with MSF4J, making it easy to bring in other 3rd party libraries and wire them in at runtime. For example, those who are familiar with Spring-JPA can use the Spring-JPA library to integrate the Hibernate ORM framework with a MSF4J service. The Spring sample demonstrates this feature in action.

Swagger support

image00

Swagger is a standard, language-agnostic interface to REST APIs which allows both humans and computers to discover and understand the capabilities of the service without access to source code, documentation, or through network traffic inspection.

This feature allows you to add Swagger annotations to your microservices to enrich the Swagger definition of your service. Please note that if Swagger annotations have not been used, the feature will still generate a basic Swagger definition by going through other JAXRS annotations in your service.

The stockquote service example demonstrates Swagger annotations in action. After running that sample, you can retrieve the Swagger definition of the service by going to http://localhost:8080/swagger?path=/stockquote.

javax.ws.rs.ext.ExceptionMapper support

An ExceptionMapper maps a Java exception to an HTTP response that is to be sent back to the invoker of the MSF4J service. This feature allows Java developers to throw business logic exceptions from their MSF4J services, and if there are ExceptionMappers registered to handle such exceptions, the toResponse method in those mappers will be invoked.

image03

Directly returning Java stacktraces to clients is a bad practice. As  a best practice, a properly designed HTTP response which doesn’t expose critical internal implementation details or security details should be returned.

ExceptionMapper allows Java developers to write their code using Java best practices, i.e. throw exceptions when exceptional conditions are encountered, and also follow best practices in service design by not exposing internal details by returning stacktraces.

The stockquote service example demonstrates ExceptionMapper in action.

javax.ws.rs.core.StreamingOutput support

image02

Streaming file download

StreamingOutput gives the developer full control over how the response from your microservice is going to be streamed. You can choose the size of the chunks that are going to be streamed. Depending on the available resources to the JVM, you may select your chunk sizes. For example, smaller chunk sizes will allow you to operate with a lower server side memory footprint. However, note that this may cause the clients to experience higher latencies. The idea is that you can choose a chunk size that is a good tradeoff between memory footprint and latency.

A practical example could be streaming a large file to a client or reading a large number of records from a database and streaming data based on those records.

The FileServer example demonstrates StreamingOutput in action.

FormParam and FormDataParam annotation support

The @FormParam annotation supports HTML form submission with application/x-www-form-urlencoded and multipart/form-data MIME types. The value will be automatically converted to the corresponding parameter type and assigned to that parameter in the MSF4J service operation.

The difference between @FormParam and @FormDataParam is that FormParam can support only Java Strings and primitive types (int, long, Integer, Long etc). The @FormDataParam annotation, on the other hand, can support complex types and Collections, with multipart/form-data content type and support for files along with form field submissions. These can be directly submitted to a MSF4J service and will be handled seamlessly.

The form service sample demonstrates this feature in action.

FormParamIterator

In certain case, a user may want to have full control over the incoming message stream from form submissions, for performance optimization reasons. This can be achieved using the FormParamIterator. When a FormParamIterator is declared as the only parameter of your service operation, MSF4J will hand over full control of the stream to you. The FormParamIterator is injected via the @Context annotation.

This feature is demonstrated in the form service sample.

CircuitBreaker support

Nygard’s circuit breaker pattern defines that rather than wasting valuable resources trying to invoke an operation that keeps failing, the system backs off for a period of time, and later tries to see whether the operation that was originally failing works. This is now supported supported in MSF4J using the Netflix Hystrix library.

image04

Circuit breaker lifecycle

In the context of microservices, a typical usage of circuit breakers would be when one microservice calls another or when a microservice calls out to an external system. In such cases, these calls will be wrapped in Hystrix commands which will take care of handling the circuit breaker state management. The circuit breaker parameters can be configured as per the configurations listed athttps://github.com/Netflix/Hystrix/wiki/Configuration.

This feature is demonstrated in the MSF4J circuit breaker sample.

As you can see, we’ve added quite a bit to the second version of MSF4J. Try out MSF4J 2.0 and let us know what you think – it’s free and open source.