Exposing WSO2 BAM Statistics as a Service Using WSO2 Data Service Server
- Nadeesha Gamage
- Lead Solutions Engineer - WSO2
This article looks at how WSO2 Data Services Server can be used to expose statistics generated by WSO2 Business Activity Monitor as a web service to external service consumers.
Pre-requisites
WSO2 Business Activity Monitor
Applies to
WSO2 Business Activity Monitor |
2.3.0 or above |
WSO2 Data Services Server |
3.2.0 or above |
Table of contents
- Introduction
- WSO2 BAM architecture
- Exposing WSO2 BAM statistics via WSO2 DSS
- Scenario
- Summary
- References
Introduction
WSO2 Business Activity Monitor (BAM) is a fully open source solution that provides the capability to aggregate and analyze and present data to different business stakeholders in an organization. WSO2 BAM is a key product in the WSO2 big data story. It can provide an organization the capability of storing and processing large amounts of data. WSO2 BAM runs Apache Cassandra underneath, providing a scalable big data repository to store data from multiple sources that need to be analyzed by BAM; data analyzers powered by Apache Hadoop provides parallel, distributed data processing capabilities to WSO2 BAM. Once the data is analyzed, BAM provides the capability to present this information in the form of graphs and reports via the BAM dashboard. Similarly, WSO2 Data Services Server (DSS) can be used to expose these data to 3rd party tools, applications, and other service consumers as a web service that can be accessed via a web service invocation. This article looks at how information and statistics processed by WSO2 BAM can be exposed as a web service to be consumed by other applications in the organization’s eco system.
WSO2 BAM architecture
As depicted in the diagram above, different sources would be publishing data to WSO2 BAM, these data sources can be WSO2 products or any other external tool that can utilize the Thrift or REST endpoints exposed by WSO2 BAM.
Data receiver nodes within WSO2 BAM would receive the data and store them in a Cassandra data store. The stored data would then be analyzed by the Analyzer engine. The analyzed statistics would then be stored in a separate relational database. This relational database would act as a data source to the presentation engine of BAM and can also be a data source that can be utilized by any other external tool.
Exposing WSO2 BAM statistics via WSO2 DSS
As explained in the BAM architecture section above, BAM would store all the analyzed statistics in a relational database that can be accessed by its dashboard server. Similarly, this information can exposed as a service through WSO2 DSS. This process can be illustrated as shown in the diagram below.
Exposing BAM statistics as a service opens up more opportunities to an organization. 3rd party tools, and applications can now access these statistics as a service via WSO2 DSS. The statistics can be exposed as SOAP or REST services. Similarly, they can be throttled and exposed securely by enforcing WS-Security or Basic Authentication via WSO2 DSS.
Scenario
The scenario would include a stream of HTTPD logs sent to WSO2 BAM for analysis. WSO2 BAM would store the records in the Cassandra data store and analyze these data based on a set of Hive queries defined. Analyzed data would then be stored in the relational database that would be exposed as a service via WSO2 DSS. WSO2 BAM documentation provides a sample on HTTPD log analyzing [1] which can be used to demonstrate this scenario.
- Configure the database – By default all the samples are configured to run on internal H2 database. For this scenario, an MySQL database would be used; you can configure any JDBC database for this scenario. Update the WSO2BAM_DATASOURCE entry in the
<BAM_HOME>/repository/conf/datasources/bam-datasources.xml with the configuration below. Make sure to change the DB URL and access credentials as required.
WSO2BAM_DATASOURCE The datasource used for analyzer data jdbc:mysql://localhost:3306/BAM_STAT_DB?autoReconnect=true root root com.mysql.jdbc.Driver 50 60000 true SELECT 1 30000 - Create a database in MySQL - Create a database in MySQL. As per the above configuration the database name used is BAM_STAT_DB.
- Add the MySQL connector to WSO2 BAM and WSO2 DSS - WSO2 products are not shipped with the MySQL connector, hence you would need to add the MySQL connector jar file to repository/components/lib folder in both these products.
- Execute the sample for HTTPD log analyzing – Execute the Sample for HTTPD log analyzing found in the WSO2 BAM documentation provided here [1]. Make sure the above activities are correctly performed before executing this sample; once this is done and when the Hadoop job has been executed you can see that the data is available in the MySQL database.
- Start the WSO2 DSS server. If you are running both BAM and DSS instances in the same machine, start the DSS server with a port offset of 1.
- Upload the Carbon Application aRchive (.CAR) file available in this article, which includes all artifacts relating to the data service that is required to execute this scenario. The CAR file needs to be uploaded to WSO2 DSS, and it can be uploaded from the Carbon Application section found on the left hand navigation bar of the management console. More information on uploading a CAR file can be found in the following link [2].
-
Once the CAR file is deployed, you would see a new data service being added to the deployed services list. You can try this service from the try tool available within WSO2 DSS. This will allow you to access BAM statistics as a service from the data services server.
As BAM processes more data, statistics would be updated and would be available via this web service. This scenario describes a basic use-case on exposing BAM statistics as a service; it is possible to have multiple queries that can query multiple tables at the same time to provide more comprehensive access to statistics via a web service.
The service can be exposed as a REST or SOAP service from WSO2 DSS. Similarly, the service can be secured via WS-Security to enforce secure access to the service.
Summary
WSO2 BAM is a key product in the WSO2’s product stack; its capability to process large amounts of events and the scalability that it can provide can be useful to many organizations that are keen to build or upgrade its big data capabilities. This article demonstrated how WSO2 DSS can enhance the capabilities of WSO2 BAM by exposing statistics as a service to different service consumers who can leverage the information available in WSO2 BAM to carry out more comprehensive analysis on an organization’s activities, and thereby, take more informed decisions.