Introduction: A bit of Terminology

When hosting a cloud, you have to be aware of the multi-tenant aspect which can cause problems in consuming cloud resources. A large number of users may use the cloud environment in different ways. If your cloud offering is a PaaS, users will host services or web applications. If it is SaaS, they'll be using your software. If it is an IaaS, they'll be running VM instances in your platform. So it is really important to have reliable billing, metering and throttling mechanism within the platform.

WSO2 StratosLive is a public cloud deployment. There needs to be a way of measuring how tenants consume cloud resources. It is necessary to measure them to control or facilitate the tenants consuming resources. StratosLive encourages tenants to make full use of its resources. But there needs to be a control mechanism too. Billing, Metering and Throttling components are responsible for measuring resource usage, billing the tenants and controlling access to resources in StratosLive.

Before we begin, let's look into some terminology and product descriptions.

What is Metering?

Metering is the basis for throttling and billing. Metering measures levels of resource utilization, such as network bandwidth usage and data storage volume, consumed by the cloud services subscribers, aka tenants.

Resource utilization's (bandwidth and storage usage) are measured on the fly and the measured data is stored for summarizing and analyzing.

What is Throttling?

Throttling is the process of access and usage controlling based on a set of rules. Throttling rules are run against the metered data. Throttling rules define the upper limits for resource utilization. Access control to cloud computing services will be governed by the throttling module that evaluates throttling rules. If the current metered utilization is below the allowed upper limits, access will be granted, else revoked.

What is Billing?

Billing is the process where the cloud computing service subscribers are charged based on the metered data. Billing usually happens on a monthly basis. Billing uses pricing rates defined against usage plans and compute the invoice against the usage data.

What is WSO2 StratosLive?

WSO2 Stratos is the complete, SOA and developer platform offered as a self-service, multi-tenant, elastic runtime for cloud infrastructures. This means our complete SOA platform - now enhanced with Tomcat and Webapp support - is available as a "cloud-native" runtime that you can either use on the Web, on Amazon VPC, or on your own internal private cloud based on Ubuntu Enterprise Cloud, Eucalyptus and vmWare vSphere. Stratos is a complete, 100% open source Platform-as-a-Service for private, public and hybrid cloud deployments.

What is a Tenant ?

A tenant is like an organization (Eg: sanjeewa.com) and it has users (like departments within an organization). There is an administrator for each tenant while the others are users. The administrator allows users to use services and governs the entire system. The following diagram depicts this scenario.

The Importance of Usage and Throttling

WSO2 Stratos is the only tomcat based server which provides many enhanced features like metering, throttling and billing. A reliable billing/throttling model is crucial for cooperate service providers. WSO2 Stratos 1.5.1 provides a complete solution for Web services with pricing model. Its usage and throttling model carries out 2 main actions.

1. Before a user performs some action, checks whether he is allowed to do so (Ex adding new service/ service invocation).

2. Keeps user actions recorded (sanjeewa.com tenant used 100kb service Bandwidrh)

The following diagram depicts an overview of usage and throttling.

Usage Agent

The Usage Agent is responsible of measuring parameters and publishing them to Business Activity monitor server (Where we store data and monitor them). For each and every tenant, the agent measures some important parameters related to their usage of bandwidth and resources (Service/webApp bandwidth, Registry usage, User count). This usage metering process uses some advanced listening technologies to avoid slow downs or block entire process.

For example, say a Web service, hosted by sanjeewa.com, tries to invoke some client . This invocation takes 100kb service bandwidth. So we will store that the tenant (sanjeewa.com) got service bandwidth of 100kb at this time. That action performed by usage agent

Usage Summery Generator

Summery generator component is responsible for summarizing captured data. After collecting raw data, we sort and summarize periodically and store those records in database. Those records are available per hourly, monthly, daily, quarterly and annually basis.

According to the example given earlier, after one hour, we put the 100kb entry to the hourly table. At the end of the day, it will go to daily table with the corresponding day of the month.

Throttling Manager

Throttling is used to avoid excess usage of resources and bandwidth. Throttling manager periodically checks the usage records (data captured by the usage agent mentioned above) available, evaluates throttling rules and defines sets of actions for each and every tenant. These actions contain what tenants can or cannot do and the error messages shown to them. These rules are written in drools.

Drools is a Business rule management system (BRMS) with a forward chaining inference-based rules engine, more precisely known as a production rule system, using an enhanced implementation of the Rete algorithm. Using a rules engine can lower an application's maintenance and extensibility costs by reducing the complexity of components that implement complex business logic.

Inside WSO2 stratos throttling component, drools is used for throttling configurations and Stratos manager provides user interface to edit rules easily. After evaluating the throttling rules of each tenant, we will store the validation information (which tenant is allowed and which should be blocked) in registry with the tenant ID.

Example: Throttling manager checks the data available in monthly service bandwidth table and checks the Drool correspond to that tenant. Rules specifies the service bandwidth allowed to the user and actions to be taken when the limit is exceeded. If the particular user doesn’t exceed the limits,then the registry will be updated saying “sanjeewa.com is allowed to service invocation”. If not, the registry will be updated as blocked with blocked message saying ” you have exceeded bandwidth limits”.

Throttling Agent

Throttling agent is acting as a supervising agent which checks the actions taken by users. Basically, it checks registry entry for a given tenant (updated by throttling manager). To achieve higher performance, the throttling agent use memory implementation. It takes a copy of the registry entries and keeps them cached. Reading validation information from cache is much faster than reading registry.

For example, when some tenant/user tries to invoke a Web service, the agent checks the corresponding registry entry of the user to see weather he is allowed to take the corresponding action or not. If he can then the agent allows it; else it will prompt a message to user mentioning the issue.

Tomcat server-based throttling parameters in WSO2 Stratos

In WSO2 Stratos we are able to throttle parameters related to service and bandwidth usages. Let’s investigate in detail.

01. Registry Resource Volume

For Registry usages we can throttle following parameters

Registry incoming data actions - put/ importResource/ restore

Registry outgoing data actions - dump/ get

02. Service Bandwidth

Throttle the service incoming and outgoing message sizes to the Web Services (Axis2 Service/ JAX-WS Service/ Jar Service/ Jar Service) hosted at WSO2 Stratos Application Server.

For each incoming and outgoing message, throttling agent checks whether that tenant exceeds incoming and outgoing bandwidths. Upon verifying this it allows users to invoke services. We can use two parameters (incoming/ outgoing) related to service bandwidth for throttling rules. For this throttling agent, use Handler extended from AbstractHandler and that handler acts as a listener for service invocations.

Service incoming action

03. webApp bandwidth

Throttle the service incoming and outgoing based on message sizes to the Web Application Archive (.war) files hosted in Application server. For webApp throttling we use valve extended from CarbonTomcatValve.That valve acts as a listener and checks whether the tenant allows it or not. Here we can use two parameters (incoming/outgoing) related to webApp bandwidth for throttling rules.

webApp incoming action

webApp outgoing action

04. User Count

Number of users per given tenant.

Altogether we are able to throttle 7 actions for a given tenant, as follows.

01. Registry incoming data actions - put/ importResource/ restore

02. Registry outgoing data actions - dump/ get

03. Service incoming action

04. Service outgoing action

05. webApp incoming action

06. webApp outgoing action

07. User count per tenant

References:

1. Download stratos source and more documents available at https://wso2.com/more-downloads/stratos/

2. How to build stratos 1.5.1 source

3. Sign up for a free account at stratos private cloud

4. Metering, Throttling and Billing in Cloud Computing

5. WSO2 StratosLive

Author

Sanjeewa Malalgoda, Software Engineer, WSO2 Inc.