BLS: using WSO2 to make Switzerland’s railways work better

BLS is Switzerland’s second-largest railway company. It employs about 3000 people and runs both passenger transport trains in Switzerland and freight trains across the Alps. It owns or operates on seven major lines and also operates the standard gauge railway network of the S-Bahn Bern, which spans about 500 kilometers.

BLS_RABe_535_Loetschberger

The story starts in the 1990s, when the European Commission made railway infrastructure operators separate from train operating companies in order to create a more efficient railway network and more competition. Thus, a train operating company, such as BLS, has to now request a train path from an infrastructure operator and had to pay for this path.

In 2007, the main Swiss railway infrastructure operator had to replace its 25-year-old timetable planning system. The system had the interfaces to about 50 other systems from different railway companies. Unfortunately, there was a long delay – some ten years – and costs tripled.  But by 2015, the project was back on track, with BLS determined to finish it.

In an architectural sense, BLS realized that their product teams often may not build the best fit for a problem. There are many reasons for this – including a team being unfamiliar with the most optimal integration patterns, or a preference towards one particular middleware stack simply because they understand it better. BLS thus first devised a set of non-functional properties, relevant for describing integration problems. They then devised a decision matrix that returns a number of integration patterns for a given problem. Based on this, they devised a set of integration guidelines, including how the pattern should be implemented and what middleware was available for the purpose.

image00

They were then able to get on with the problem of integration. In the data flow structure below, BLS needed to introduce a mediation component, with traceability, routing, data validation, data transformation and protocol changes as its key functionality.

image02

For this they selected WSO2 Enterprise Service Bus; with it they were able to separate transaction data from master data. Transported by the interfaces between the train operating company and the infrastructure manager are train paths and data about the network, train paths, and junctions. Data was sent as a push with the transaction data; by using WSO2 Data Services Server, they implemented a data pull to store this data as a copy in the system.

This project commenced in 2013, when BLS started evaluating products for the task. By December 2014, BLS had four products on their list: after a cost-benefit evaluation, they were down to two by January 2015, and after a successful proof-of-concept build they had selected WSO2 by April 2015.

In their talk at WSO2Con EU 2015, the BLS executives described themselves as being satisfied with WSO2 on many fronts, both product release schedules and financial growth; the availability of partners in Switzerland; with the architecture and cost effectiveness of the product; and also, with the availability of the source code. Using WSO2’s Quick Start Program, they were able to rapidly prototype cost-effective solutions for their integration.

At WSO2, we’re proud to be a part of BLS’s success. Our open source products are used by enterprises around the world – ranging from companies like BLS to governments. If your organization has a need for world-class middleware, talk to us. We’ll be glad to help.

WSO2 API Manager 2.0.0: Better Statistics

Introduction

Any API management tool requires the ability to view statistics – accurately and with details essential to the operation of APIs. WSO2 API Manager 2.0.0 introduces powerful analytics features along these lines by using the capabilities of WSO2 Data Analytics Server (WSO2 DAS), bringing all API, application and subscription related statistics to one location.

image05

New additions

In order to help you make sense of these analytics, we’ve added the following graphs:

1.Published APIs Over Time

This graph can be used to check API creation rates. A user can check APIs based on the API provider or can view statistics related to all API providers.

image05

2. API Latency

The API Latency breakdown graph can be used to check time consumption for each level in the message flow. It shows total time taken to handle a request and provides the time each component spends on that request – for example, a user can check the amount of time spent on the authentication process and so on. This graph is useful for identifying delays in the API.

image013. API Usage Across Geo Locations

This graph provides statistics based on the location of the API request. It uses X-Forwarded-For header to identify the location of the user.

image06

4. API Usage Across Usage Agent

This graph provides information related to the access mechanism of the API, and helps provide insight into the kind of devices and platforms used to access the API – mobiles, computers and so on.

image08

We’ve also added an Applications Created Over Time graph under the Applications section. As with the API creation graph, this provides statistics related to application creation. Users can drill down the data based on the registered user and the API in question.

image02

In addition to these come the subscription graphs: Developer Signups Over Time and Subscriptions Created Over Time, both of  which are self-explanatory.

image00

image07

How These Statistics Work

WSO2 Analytics Server for API Manager is essentially a fully functional WSO2 DAS server.

image03

In a nutshell, WSO2 API Manager 2.0.0 sends the following event streams to API Manager Analytics:

org.wso2.apimgt.statistics.request:1.1.0

org.wso2.apimgt.statistics.response:1.1.0

org.wso2.apimgt.statistics.fault:1.0.0

org.wso2.apimgt.statistics.throttle:1.0.0

org.wso2.apimgt.statistics.workflow:1.0.0

org.wso2.apimgt.statistics.execution.time:1.0.0

org.wso2.analytics.apim.alertStakeholderInfo:1.0.0
The APIM Analytics Server process these events and writes the summarized data to a shared database. WSO2 API Manager then queries statistics from this database to display in the API Manager Publisher and Store.

Footnotes

One significant change in WSO2 API Manager 2.0.0 statistics is the removal of the graphical user interface (GUI) for configuring statistics. In WSO2 API Manager 1.10, a GUI was provided to handle this DAS-related configuration. We’ve moved this functionality back into the api-manager.xml file.

We’ve also taken out the DAS Rest client – with 2.0.0, the default and only implementation is APIUsageStatisticsRdbmsClientImpl, which uses an RDBMS client to get data from the database.

Since this is essentially a DAS server, you can use features such as gadgets and realtime analytics (using Siddhi) for better or more complex visualizations of the events sent from WSO2 API Manager.

Keep your WSO2 products up to date with WUM

We at WSO2 continuously improve our products with bug fixes, security fixes , and various other improvements. Every major release of our WSO2 products is followed by a series of periodic dot releases that roll up all these recent updates.

We want everyone evaluating, developing on, or preparing WSO2 products for production deployments to have the best experience and especially not trip over a bug that has already been fixed! WSO2 Update Manager (WUM) makes this process much easier – now you can get access to latest updates as and when we release them.

What is WUM?

WUM is a command-line application which allows you to keep WSO2 products up-to-date. It determines which updates are new and relevant for your WSO2 products, downloads and installs them. WUM builds you a new product distribution with all the relevant updates; this distribution is ready to deploy into your development or production environment. Note that WUM will not push updates into production: you have full control over when and how you want to such pushes to occur.

Updates are available for the following products and versions. We are in the process of adding more products and versions.

  • WSO2 Enterprise Service Bus 5.0.0
  • WSO2 Enterprise Service Bus 4.9.0
  • WSO2 Enterprise Service Bus Analytics 5.0.0
  • WSO2 API Manager 2.0.0
  • WSO2 API Manager Analytics 2.0.0

Note that fresh WSO2 product distributions downloaded via the wso2.com site or using WUM, are licensed under Apache 2. However, product distributions built by WUM with updates installed are licensed under WSO2 Update License 1.0. The updated product distributions are not licensed for use in production without a valid WSO2 subscription. Visit http://wso2.com/support for more information about WSO2 subscription.

Installing WUM

You can download WUM from http://wso2.com/update. Download the binary release suitable for your operation system and processor architecture. Binary distributions are available for Linux, Mac OS, and Windows (follow installation instructions available on the same download page).

Once the installation is completed, open a new terminal and execute the following command. You should get the help page with all the available commands.

$ wum

Initialize WUM with your WSO2 credentials

You need to have a WSO2 account to get started with WUM. If you don’t have an account with WSO2, please create one here.

$ wum init
You need a WSO2 account to start using wum.
Don't have one yet? Sign up at https://wso2.com/user/register

Please enter your WSO2 credentials to continue
Username: sameera@wso2.com
Password for 'sameera@wso2.com': 
Authenticating...
Done!

-- Welcome to WUM 1.0-beta --

* Find more information at http://wso2.com/update
* Documentation: https://docs.wso2.com/display/ADMIN44x/Updating+WSO2+Products
* JIRA: https://wso2.org/jira/browse/WUM

What's next? Have a look at the following list of wum commands:

Add WSO2 products to your product repository
  wum search  Search WSO2 products         
  wum add     Add or download WSO2 products

Update WSO2 products available in your product repository
  wum check-update Check for new updates    
  wum update       Update your WSO2 products

Manage WSO2 products available in your product repository
  wum list      List WSO2 products           
  wum describe  Show details of WSO2 products
  wum delete    Delete WSO2 products

Use "wum [command] --help" for more information about a command.

Use "wum [command] --help" for more information about a command.

Local product repository

WUM maintains a local product repository in your machine where it stores WS02 product distributions. Whenever you update a product, WUM creates a new distribution with all these updates and stores it in this repository. By default, WUM creates this repository in the user’s home directory, but you have the option to change this as well.


Search and add WSO2 products

Before you update any WSO2 product, you need to add the product to your local product repository. You can either download a product or add an already downloaded product. If you are not sure about what product name to use, you can use the ‘wum search’ command. If you run the search command without any arguments, it will list all the WUM supported WSO2 products.

$ wum search
Connecting to WSO2 Update...

wso2esb-5.0.0 - Enterprise Service Bus
wso2esb-analytics-5.0.0 - Enterprise Service Bus Analytics
wso2am-2.0.0 - API Manager
wso2am-analytics-2.0.0 - API Manager Analytics
wso2esb-4.9.0 - Enterprise Service Bus

What's next?
  use "wum add <product>" to download a product
  e.g "wum add wso2esb-5.0.0" to download Enterprise Service Bus 5.0.0

You can also search using keywords.

$ wum search api manager
Connecting to WSO2 Update...

wso2am-2.0.0 - API Manager
wso2am-analytics-2.0.0 - API Manager Analytics

What's next?
  use "wum add <product>" to download a product
  e.g "wum add wso2am-2.0.0" to download API Manager 2.0.0

If you look at the examples given in the “what’s next” section, you would see the expected format of the product name to use. Use the following command if you want to add the latest version of wso2esb.

$ wum add wso2esb
Connecting to WSO2 Update...
The following product(s) will be downloaded.
  wso2esb-5.0.0.zip
After this operation, 218.1MB of additional disk space will be used.
Do you want to continue? [Y/n]
Downloading product wso2esb-5.0.0... [2.4MB/218.1MB]

If you want to add a specific version of wso2esb then you need to specify the version as follows:

$ wum add wso2esb-4.9.0

You also have the option to add an already downloaded product distribution using the add command.

$ wum add --file ~/user/wso2/dist/wso2esb-5.0.0.zip

Now try the following command to see the list of add products.

$ wum list

Update WSO2 products

As I’ve explained earlier, WUM creates a new product zip file with all the updates installed. But if you want to check whether updates are available before you attempt to update, you can use the “check-update” command as follows.

$ wum check-update wso2esb-4.9.0
Connecting to WSO2 Update...

Checking for latest updates for wso2esb-4.9.0...
96 updates are available
[WARNING] 11 critical security updates. WSO2 strongly recommends that you install these updates now.

What's next?
  use "wum update" to install latest updates

Now let’s update the products using the “update” command

$ wum update wso2esb-4.9.0
Connecting to WSO2 Update...

Validating your subscription status for product wso2esb...
[WARNING] Your credentials are not associated with an active WSO2 subscription.
Please remember that updates are not licensed for use in production without a valid WSO2 subscription.
See: http://wso2.com/update for more details.

Updating wso2esb-4.9.0...
96 updates are available
Downloading updates...
Installing updates...
Preparing update summary...
Building updated product...
Update summary:
  Installed updates: 96
  * [WARNING] WSO2 strongly recommends to use this updated product for production as soon as possible.
  Security updates: 11
  Updated Product: /Users/sameera/.wum-wso2/products/wso2esb/4.9.0/wso2esb-4.9.0.1472889183206.zip
  * More information about updates are available inside the above product archive
  Update summary(pdf): (product-archive)/updates/summary-2016-09-08T19-23-50/update-summary-1472889183206.pdf
  * Configuration changes are available
  Change log: (product-archive)/updates/summary-2016-09-08T19-23-50/change-log-1472889183206.txt

What's next?
  use "wum list [<product-pattern>]" to list products in your local repository
  use "wum describe [<product-pattern>]" to get more details of products

If you don’t have an active WSO2 subscription for wso2esb, you would get a warning as above. This means you can use the updated product for your development activities, but you are not supposed to use it in production.

Once the update is completed, you should be able to see the updated product files in your local repository. We version updated product distributions using the timestamp of the latest update.

$ wum list
Product        Updated              Filename                       
wso2am-2.0.0   08 Sep 16 18:01 IST  wso2am-2.0.0.1473322072976.zip 
wso2am-2.0.0   -                    wso2am-2.0.0.zip               
wso2esb-4.9.0  08 Sep 16 19:24 IST  wso2esb-4.9.0.1472889183206.zip
wso2esb-4.9.0  -                    wso2esb-4.9.0.zip

Manage your product repository

There are several WUM commands available for you to manage your local product repository. The “wum list” command lists all the available products. If you want to get more information about a product, you can use the “wum describe” command. It gives you the location of the product file and update history, among other information.

$ wum describe wso2esb-4.9.0.1472889183206.zip
Filename:  wso2esb-4.9.0.1472889183206.zip
Product Name:  wso2esb
Product Version: 4.9.0
Kernel Version:  4.4.1
Last Updated Time: 2016-09-08T19:23:55+05:30
Product File Path: /Users/sameera/.wum-wso2/products/wso2esb/4.9.0/wso2esb-4.9.0.1472889183206.zip
No Of Times Updated: 1
Installed Updates: 96
Update History:
 Date                       Installed Updates  Username        
 2016-09-08T19:23:55+05:30  96                 sameera@wso2.com

In case you want clean up your product repository by deleting files, you can use the “wum delete” command.

$ wum delete wso2esb-4.9.0
The following product file(s) will be deleted.
  wso2esb-4.9.0.1472889183206.zip
  wso2esb-4.9.0.zip
Do you want to continue? [y/N]

The above command will delete all the wso2esb-4.9.0 product files including the updated files. List, describe and delete commands accepts a product pattern; therefore you can filter out products by given a particular pattern.


We’ve released the Beta version of WUM. Try it out and let us know what you think.

CREATE-NET Discusses WSO2 and the Future of IoT

Charalampos Doukas is a researcher at CREATE-NET – or rather, the Center for Research And Telecommunication Experimentation for NETworked communities, the non-profit research center headquartered in Italy. Charalampos spoke at WSO2Con EU 2015 about his research into the world of open source in IoT and where WSO2 stands in this context.

In his 28-minute presentation, Charalampos started off by pointing out that despite the strange lack of discussion about open source in IoT conferences, to him the whole thing started with the open source community “with people connecting their Arduinos to the Internet and sharing their sensor data.” In fact, Pebble and SmartThings (the smart home platform maker acquired by Samsung) both used Arduinos for their 2012 proofs of concept; open source has always been closely tied to IoT platforms as we know them.

From a developer’s perspective, an IoT platform must be able to connect devices to each other and to users and to allow services to consume the data and control these devices, delivering interesting use cases. The main features, thus, are to communicate with and actuate devices, to collect and manage data from them, and to allow user interaction. A “spaghetti” of standardization bodies push a wide variety of protocols and standards for doing all this.

As Charalampos explained, there are over 40 IoT platforms that fulfill these requirements. Some of them, like ThingSpeak and Nimbits, are open-source; Nimbits, one of the oldest, runs on Google App Engine and even integrates with Wolfram Alpha (leading to some interesting use cases). Then there are the likes of SiteWhere, which embeds WSO2 Siddhi for Complex Event Processing and connects to WSO2 Identity Server.

“So, WSO2,” he said in his talk. “This picture is quite clear and illustrates the different layers that you need to build an IoT application and where WSO2 starts. You have the devices, you have the enterprise service bus, and message broker that enable the messaging; you can do the processing and analytics, and on top of that you can have things like a dashboard or web portal for managing data and devices. The new things that are coming – and hopefully will be more and more improved and used – are the device manager and identity server.”

Charalampos quickly sketched out what he sees as the core components of the WSO2 IoT platform: the WSO2 Message Broker, Enterprise Service Bus, Identity Server, Enterprise Mobility Manager, User Engagement Server, API Manager, Business Activity Monitor and Complex Event Processor. Yes, it’s a handful to enunciate – but the way we’ve built our platform, each component is built on the Carbon framework and provides functionality that you can add and subtract as needed. This makes it easy to not just maintain the lightweight stack that an IoT solution typically needs, but also to integrate with other software that provides similar functionality.

One of our biggest changes since then is to create an all-new product, the upcoming WSO2 IoT Server, bringing together the best of the WSO2 platform’s many capabilities into a more out-of-the-box, enterprise-grade server-side IoT device management architecture. Once integrated with WSO2 Data Analytics Server (which contains the functionality of WSO2 CEP and WSO2 BAM), it offers advanced IoT device analytics, including policy-based edge analytics and predictive analytics using machine learning. And true to the roots of IoT, this remains open source.

To explore this future addition to our IoT platform for free, visit wso2.com/products/iot-server/. To watch the full video of Charalampos Doukas’ analysis of the IoT sphere, click here.