2014/09/08
8 Sep, 2014

Exposing WSO2 BAM Statistics as a Service Using WSO2 Data Service Server

  • Nadeesha Gamage
  • Lead Solutions Engineer - WSO2

This article looks at how WSO2 Data Services Server can be used to expose statistics generated by WSO2 Business Activity Monitor as a web service to external service consumers.

Archived Content
This article is provided for historical perspective only, and may not reflect current conditions. Please refer to relevant product page for more up-to-date product information and resources.

Pre-requisites

WSO2 Business Activity Monitor

WSO2 Data Services Server

Applies to

WSO2 Business Activity Monitor

2.3.0 or above

WSO2 Data Services Server

3.2.0 or above

Table of contents

  • Introduction
  • WSO2 BAM architecture
  • Exposing WSO2 BAM statistics via WSO2 DSS
  • Scenario
  • Summary
  • References

Introduction

WSO2 Business Activity Monitor (BAM) is a fully open source solution that provides the capability to aggregate and analyze and present data to different business stakeholders in an organization. WSO2 BAM is a key product in the WSO2 big data story. It can provide an organization the capability of storing and processing large amounts of data. WSO2 BAM runs Apache Cassandra underneath, providing a scalable big data repository to store data from multiple sources that need to be analyzed by BAM; data analyzers powered by Apache Hadoop provides parallel, distributed data processing capabilities to WSO2 BAM. Once the data is analyzed, BAM provides the capability to present this information in the form of graphs and reports via the BAM dashboard. Similarly, WSO2 Data Services Server (DSS) can be used to expose these data to 3rd party tools, applications, and other service consumers as a web service that can be accessed via a web service invocation. This article looks at how information and statistics processed by WSO2 BAM can be exposed as a web service to be consumed by other applications in the organization’s eco system.

WSO2 BAM architecture

BAM Architecture

As depicted in the diagram above, different sources would be publishing data to WSO2 BAM, these data sources can be WSO2 products or any other external tool that can utilize the Thrift or REST endpoints exposed by WSO2 BAM.

Data receiver nodes within WSO2 BAM would receive the data and store them in a Cassandra data store. The stored data would then be analyzed by the Analyzer engine. The analyzed statistics would then be stored in a separate relational database. This relational database would act as a data source to the presentation engine of BAM and can also be a data source that can be utilized by any other external tool.

Exposing WSO2 BAM statistics via WSO2 DSS

As explained in the BAM architecture section above, BAM would store all the analyzed statistics in a relational database that can be accessed by its dashboard server. Similarly, this information can exposed as a service through WSO2 DSS. This process can be illustrated as shown in the diagram below.

Exposing BAM Stats as a service

Exposing BAM statistics as a service opens up more opportunities to an organization. 3rd party tools, and applications can now access these statistics as a service via WSO2 DSS. The statistics can be exposed as SOAP or REST services. Similarly, they can be throttled and exposed securely by enforcing WS-Security or Basic Authentication via WSO2 DSS.

Scenario

The scenario would include a stream of HTTPD logs sent to WSO2 BAM for analysis. WSO2 BAM would store the records in the Cassandra data store and analyze these data based on a set of Hive queries defined. Analyzed data would then be stored in the relational database that would be exposed as a service via WSO2 DSS. WSO2 BAM documentation provides a sample on HTTPD log analyzing [1] which can be used to demonstrate this scenario.

  1. Configure the database – By default all the samples are configured to run on internal H2 database. For this scenario, an MySQL database would be used; you can configure any JDBC database for this scenario. Update the WSO2BAM_DATASOURCE entry in the <BAM_HOME>/repository/conf/datasources/bam-datasources.xml with the configuration below. Make sure to change the DB URL and access credentials as required.
    
    			WSO2BAM_DATASOURCE
    			The datasource used for analyzer data
    			
    				
    					jdbc:mysql://localhost:3306/BAM_STAT_DB?autoReconnect=true
    					root
    					root
    					com.mysql.jdbc.Driver
    					50
    					60000
    					true
    					SELECT 1
    					30000
    				
    			
    		
    
  2. Create a database in MySQL - Create a database in MySQL. As per the above configuration the database name used is BAM_STAT_DB.
  3. Add the MySQL connector to WSO2 BAM and WSO2 DSS - WSO2 products are not shipped with the MySQL connector, hence you would need to add the MySQL connector jar file to repository/components/lib folder in both these products.
  4. Execute the sample for HTTPD log analyzing – Execute the Sample for HTTPD log analyzing found in the WSO2 BAM documentation provided here [1]. Make sure the above activities are correctly performed before executing this sample; once this is done and when the Hadoop job has been executed you can see that the data is available in the MySQL database.
  5. Start the WSO2 DSS server. If you are running both BAM and DSS instances in the same machine, start the DSS server with a port offset of 1.
  6. Upload the Carbon Application aRchive (.CAR) file available in this article, which includes all artifacts relating to the data service that is required to execute this scenario. The CAR file needs to be uploaded to WSO2 DSS, and it can be uploaded from the Carbon Application section found on the left hand navigation bar of the management console. More information on uploading a CAR file can be found in the following link [2].
  7. Once the CAR file is deployed, you would see a new data service being added to the deployed services list. You can try this service from the try tool available within WSO2 DSS. This will allow you to access BAM statistics as a service from the data services server.

    As BAM processes more data, statistics would be updated and would be available via this web service. This scenario describes a basic use-case on exposing BAM statistics as a service; it is possible to have multiple queries that can query multiple tables at the same time to provide more comprehensive access to statistics via a web service.

    The service can be exposed as a REST or SOAP service from WSO2 DSS. Similarly, the service can be secured via WS-Security to enforce secure access to the service.

Summary

WSO2 BAM is a key product in the WSO2’s product stack; its capability to process large amounts of events and the scalability that it can provide can be useful to many organizations that are keen to build or upgrade its big data capabilities. This article demonstrated how WSO2 DSS can enhance the capabilities of WSO2 BAM by exposing statistics as a service to different service consumers who can leverage the information available in WSO2 BAM to carry out more comprehensive analysis on an organization’s activities, and thereby, take more informed decisions.

References

  1. https://docs.wso2.com/display/BAM241/Analysing+HTTPD+Logs
  2. https://docs.wso2.com/display/DSS310/Deploying+Carbon+Applications
 

About Author

  • Nadeesha Gamage
  • Lead Solutions Engineer
  • WSO2