Business Activity Monitoring - An Introduction
By Samisa Abeysinghe
- 9 Feb, 2010
Business Activity Monitoring (BAM) is a term coined by Gartner. The term reflects the interests in the strategic concepts of the zero-latency enterprise and straight through processing. The gap between when the data are collected and when they are available for decision making is defined as latency in this context. Hence, zero latency means that data is immediately available. This enables organizations to be proactive rather than be reactive. Straight through processing is where inefficient steps like manual entry have been eliminated.
To make more informed and better decisions, decision makers need the ability to analyze current data as well as historical trends and situations. Both business and IT domain experts want right information at the right time and in the right place to monitor and understand business activities within an enterprise.
Need for Business and IT Alignment
Business domain experts need to introduce new business processes and fine tune existing processes, in sync with business dynamics. However, historically, it has been a problem, that even tough IT has been deemed to automate and improve efficiencies, making changes to software systems as and when necessary was not that simple. The root cause of this problem has been that the IT systems tend to be more inclined to technical domains, and they require IT experts to operate, change and improve. This made the IT systems distant from business experts and business process requirements, resulting in inconsistent processes across departments, and lack of business process visibility across the enterprise.
In order to ensure smooth business operations powered by IT systems, the business aspirations of the business domain experts and the technical capabilities of the IT systems need to match somehow. When the business process requires change, the ability to reflect those with IT systems, need to be seamless, without manual intervention by IT experts. In fact, the ideal scenario is where the business experts themselves can configure the system, and effect the changes, rather than having to talk to IT domain experts. In other words, business users should be capable of self serving the business needs through the enterprise systems.
Business and IT Alignment with SOA
Service Oriented Architecture (SOA) principles open up the opportunity to bridge the business and IT gap.
SOA help us get over the problems related to business and IT gaps, thanks to the principle of service composition. In service composition, the lowest level is the fine grained services. These services often provide connectivity services, by wrapping existing computing assets such as legacy applications, or packaged line of business applications. These services can also provide services based on data models stored either in databases or file systems. Services that wrap data models are often called data services.
Connectivity services and data services form the finest grained level of services in service composition that can be used as the basis of the coarse grained services in the IT domain. While fine grained services are more close to implementation techniques and technologies, coarse grained services composed out of fine grained services are more close to business semantics. Both the fine grained services and coarse grained value added services could be maintained by IT domain experts, as they are more close to IT domain operations.
At the next level, business experts can take over from IT experts to define the business processes of the organization, based on the coarse grained services, without the intervention of IT experts. If the coarse grained services do not match the business aspirations, then those need to be fixed, and IT domain experts would be responsible for getting those fixed. This is where the A in SOA comes into play. The architecture must be designed to live up to service oriented principles, where basic services that act as building blocks can be used to compose business processes. Once those coarse grained services can meet the business aspirations, the mixing and matching of those services by the business experts at the next level to define business processes becomes possible.
Once the business processes are in place, the changes done to the business processes would be reflected into the applications that use the services. Hence the day-to-day business will continue to operate smoothly. And more importantly, the business will continue in sync with the decisions and tuning done by the business experts, in real time. SOA paves the way for business experts to control the business processes the way they want, leveraging the power of the underlying IT systems.
Business Process Management (BPM)
Business Process Management (BPM) is about process modelling, execution and management. You can achieve continuous process improvement by constantly monitoring the situation and taking corrective action. For this, it is required to get business and IT working together to achieve business objectives, making IT more responsive to changing business needs and conditions.
The mere fact that SOA makes business and IT to align, does not make it solve all the problems. To stay competitive and reap the benefits set forth by SOA, it is a must that the trends be constantly monitored and decisions made and actions taken on a regular basis to fine tune business processes. It is also required to get IT to respond as and when needed.
For this, in addition to those tools that help us orchestrate business processes, we also need tools to effectively monitor the state of business powered by the underlying SOA. In other words, it is a feedback cycle that needs to be in place, and not a static service deployment.
For example, as portrayed by the above figure, ESB connects the connectivity services and data services in the enterprise SOA deployment. The service bus being the central place via which services are connected, it becomes the natural one-stop-shop choice for monitoring business activities. However, it must also be noted that individual services can too be directly monitored using a business activity monitor. To do this, the service hosting environment should have the collect and publish data into business activity monitor.
The business activity monitor can be the source of informed decision making. BAM tools can reveal current trends based on the latest data as well as historical trends based on historical data, hence can be useful in both tactical as well as strategic decision making.
Real time information can come in handy both in IT as well as business domains. For example, a noticeable drain in a service’s response count could be detected and be traced to relate to a hardware failure in the machine hosting the service or a software malfunction due to resource outage.
Likewise an increased number of requests could be traced to relate to a spike in market activity due to some external factors, and could be exploited to the maximum by increasing discounts and increased supply to keep up with the demand. Hence, with BAM the just-in-time business capabilities of an enterprise are greatly enhanced. And so is the responsiveness of the IT operations. Overall, the increase in the flexibility and responsiveness enabled by BAM makes the enterprise more agile.
Decision making still lies mainly in the hands of humans. In other words, decision making is still a largely human responsibility, and cannot be delegated to machinery or software, no matter how advanced the software and hardware systems are. However, it is obvious that, IT tools can help a great deal. For example, for human decision makers a picture is worth a thousand numbers, and hence a graph based on numbers crunched by the computer is very handy to empower the accurate and quick decision making.
Given this factor, graphical representation of the trends is a must with BAM tools to facilitate easy, precise as well as faster decision making.
When monitoring a SOA deployment, we need to clearly map the business processes and the activities that those business processes are composed of, to the services and operations that the SOA is composed of. The fact that there is nice alignment between business and IT domain in a SOA setup makes this mapping simpler.
To elaborate on this mapping further, let’s take an order processing application as an example. The following diagram shows the business process involved.
Note that, when the above business process is in execution, not all orders placed would end up with successful closure with a payment. That is the reality of any real world business. Along the business process, due to various reasons such as, delays, non-availability of stocks, technical failures and the like, various activities would not succeed. Overall it is usually a funnel pattern that can be observed from the activities towards the start of the business process to the activities towards the end of the business process. Hence a useful and easy to monitor visualization of the business process would be to use a funnel chart, and monitor the business success ratio.
The business domain experts could set forth various key performance indicators (KPI), to help monitor the success of the business flow. For example, if the payments as a ratio of the orders placed are greater than 60%, it would be considered a successful business. In the short term, if this falls below expected level, immediate investigation to identify the cause need to be carried out and based on the outcomes, actions taken.
The reasons for momentarily falling short of expectations could be due to an operational glitch on the part of IT, like machines running out of memory. For example, the invoicing service might have become slow due to hardware resource limitations, causing payments to get delayed, as clients would not have been sent invoices on time. So rather than business domain personnel, IT need to spring into action here and make sure the situation is brought back to normal. The hardware resourcing might have to be provisioned accordingly and decisions be made to go for a more scalable infrastructure.
The problems causing the dropped number of completed orders could also be due to business aspects. For example, over pricing compared to a competitor, a competitor running a promotional campaign, or delays in fulfillment process due to various procedures in place in the stores, etc. So the funnel based monitoring can alert the business personnel on the current state of the business, and persuade them to proactively dig into the root causes of the problems.
The funnel tool can also be used in strategic decision making to expand the business. For example, in order to figure out how to increase the order completion percentage, the funnel could be used to monitor the bottlenecks over time. At which point, in the business funnel, the greatest number of drops happen for the orders placed, without proceeding to next stage. Starting from the point of placing the orders, the funnel chart could be analyzed to identify the success and failure patterns, and then dig into the problems, in search of means to improve successful conversion ratio from one step to the next.
The business funnel is only one example out of numerous monitoring tools that can be used to detect patterns and trends, make decisions and take action both in business and IT domains. The BAM tool that you choose would offer you various options and tools in terms of visualizing and monitoring various KPIs to suite your needs. You can use a combination of those visualization tools and reports to monitor short term states as well as long term trends, and based on those, make well informed tactical as well as strategic decisions.
WSO2 Business Activity Monitor
WSO2 Business Activity Monitor is an open source tool that can be used to monitor business activities in your enterprise. Often times, the BAM tools are bulky and expensive. WSO2 BAM is a lean product that can be used to monitor any SOA or non-SOA deployments. Given that it is completely free and open source, you can use it to try out BAM on your SOA setup.
WSO2 BAM has built in support to monitor WSO2 enterprise middleware platform products. It also has an extensible model, where any non-WSO2 setups, such as .NET deployments or Apache Axis2 based deployments could be monitored with.
WSO2 BAM can be run as a stand alone server, and you can use a simple event driven model to publish data into the BAM server and then visualize those with multiple dashboards. The main dashboard supports gadget technology, where the users can write their own gadgets to cater their unique monitoring needs.
Samisa Abeysinghe, Director - Engineering, WSO2 Inc., samisa AT wso2 DOT com.