Throttling in a cloud computing environment
- Amila Mahaarachchi
- Associate Director / Architect - WSO2
Introduction
Cloud computing is becoming more and more popular in the IT world these days. Cloud solutions providers come up with a lot of cloud implementations. People try out these cloud platforms and people who are convinced, start using the cloud in their enterprises too. Cloud vendors are concerned about their cloud’s smooth functionality and high up time. They have to keep an eye one how users are consuming the cloud resources. Throttling is the solution for controlling how users are allowed to consume the cloud resources. In the rest of this article, you will find out more on throttling and how to use the throttling functionality available in WSO2 StratosLive, the complete Open PaaS.
- What is throttling
- Why throttling is important in cloud computing
- What can be throttled
- Throttling in WSO2 StratosLive
- How to configure throttling using WSO2 StratosLive
- On a LAN, the network admin may throttle the bandwidth to control network congestion and server crashes.
- An ISP may throttle the bandwidth for some users to control their usage above a certain limit.
- CPU throttling is used to conserve power or reduce the heat generation.
Why throttling is important in cloud computing
When hosting a cloud, you have to be aware of the multi-tenant aspect, which can cause problems in consuming cloud resources. A large number of users may be using the cloud environment in different ways. If your cloud is a PaaS, they will be hosting services or web applications. If it is SaaS, they’ll be using your software. If it is an IaaS, they'll be running VM instances in your infrastructure.
In any of the above scenarios, if a user is consuming too much of your resources, it can cause problems to the entire cloud environment. Either you have to control the user's behaviour or you have to scale up your resources and charge the user for consuming resources. If you decide to scale up and charge, it is always good to have a control mechanism.
Another advantage of throttling is, you can use it to support your billing system. For example, if you charge based on subscriptions and you want to restrict the users consuming resources not allowed by their subscription, then, throttling is the answer for you.
In a cloud environment, there are several parameters which can be throttled.
- Network bandwidth
- Storage
- CPU usage
- I/O operations
Lets see how throttling is used by a real cloud environment. I am considering WSO2 StratosLive, the complete, 100% Open Source PaaS from WSO2, to describe how throttling is used.
Throttling in WSO2 StratosLive
WSO2 StratosLive presents the complete middleware platform from WSO2 known as “Carbon” in the cloud. Users can create tenants and use the StratosLive cloud environment. It uses throttling to ensure the users only consume resources which are allowed to them according to the usage plan they subscribe to. It throttles following parameters.
- Webapp bandwidth - Tenants can host webapps in WSO2 Application Server. When requests are received and responses are sent, it is checked whether the tenant has permission to consume the bandwidth. If not, i.e. tenant has already consumed the allocated bandwidth, the request will be rejected.
- Service bandwidth - Same as webapp bandwidth. This applies to the services hosted in StratosLive.
- Registry bandwidth - This is considered when registry items are accessed by tenants. When a get/put operations is carried out in the registry, a certain bandwidth is consumed. It also undergoes throttling.
- Registry storage - Tenants are allowed to store artifacts in the registry. But to control the storage consumed, it is throttled.
- Number of users per tenant - When a tenant is created, tenant admin can add users to the tenant. This can be throttled if needed.
Above mentioned throttling is done based on monthly usage data of tenants. Not only that, there are throttling rules. These rules can be changed on the fly, making WSO2 StratosLive's throttling capabilities very powerful.
How to configure throttling using WSO2 StratosLive
Next, let us take a look at how to set up WSO2 StratosLive with throttling. We mainly focus on setup and fine-tuning parameters for throttling and usage metering. First, you need to understand few terms.
Throttling Manager – Manage, evaluate what users can and cannot do based on their subscription plans and their current usage.
Throttling Agent – This component detects user actions and allows or blocks it based on the validation information generated by throttling manager
Usage Plan- There are 4 types of usage plans available at WSO2 StratosLive. According to usage plan, users will allow different volume of resources and service usage. (Demo/SMB/Professional/Enterprise). You can find more information on usage plans of WSO2 StratosLive from here.
We have to configure few things before it works properly.
1. How often throttling manager executes. Throttling manager periodically evaluates the tenants usage of resources (Service/webApp bandwidth, Registry usage, User count) and check it with their maximum allowed limit. After that process, it stores this validation data on registry. Here we can configure when to start this evaluation process and how often we do this by editing the configuration file, throttling-config.xml. It is assumed that you have setup Stratos correctly in your environment. Then go to manager/repository/conf folder. There you will see throttling-config.xml file and that file has following entries.
interval parameter is there to specify the time frequency (in minutes) of executing throttling manager. According to this configuration, it executes rules every 60 minutes. That means, Configuration Manager detects any excess usage of resources within one hour. After each 60 minutes, it calls Data services, gets current usage of each tenant and updates their actions (What they are allowed to do or not)
delay parameter is there to define when to start first rule execution. According to this configuration, server runs the first throttling rule execution 15 minutes after the server started.
15 and 60 minutes are recommended values. But if you need much more accuracy you can reduce that interval value (But its advisable to use some value more than 3 minutes; else it causes performance drop). Also, remember that reducing this time always causes higher CPU Usage. If you don't need to run throttling manager you can disable it by setting a negative value for interval parameter. If you plan to run a manager instance without throttling you can use this configuration.
2. How other Services allow Throttling Manager usage. In order to use throttling manager by services, we have to modify configuration in each and every service that we use (Appserver/ IS/ BAM/ DSS etc..). Those services need to access throttling manager at WSO2 Stratos Manager. In that case we have to modify throttling-agent-config.xml file, which has following content.
managerServiceUrl – is the URL that manager runs on - Manager services URL. You will see this URL when manager starts up at terminal. It is set to local host by default.
userName - Here we have to specify Stratos Manager server super admin user name (By default it is admin)
password – Specify Stratos Manager server super admin password (By default this will be admin)
This configuration is used by other services to invoke the throttling rules at manager. For example, when we add new user, the identity server will call throttling manager and invoke throttling rules for the respective tenant. Otherwise within one hour of throttling, rule execution time tenant user may add any number of users. To avoid such misuses we use this technique.
3. How to write throttling rules. For each usage plan we have to specify the upper limit of resource and service usage. If any user tries to go beyond that limit, throttling agent will stop that action immediately. For example, if Demo account is allowed to add 1 users we have to specify it in throttling rules. You will see throttling-rules.drl file inside manager/repository/conf.
At first, manager takes rules from that file and stores it in registry. Only super admin user for Stratos manager is allowed to change throttling rules. If you logged in as super Admin you will see the throttling menu. By clicking it you can see the throttling rules editor window where you can write rules.
References:
2. How to configure throttling in Cloud Environment using WSO2 Stratos – Users Guide
3. Metering, Throttling and Billing in cloud