2009/12/08
8 Dec, 2009

WSO2 Throttling

  • Isuru Suriarachchi
  • Technical Lead and Product Manager - WSO2

If you have developed Web services, have you ever thought about how to control access to your Web service remotely? This article explains how you can do that using WSO2 Throttle module.

Table of Contents

Introduction

When Web services are used at the enterprise level, it's a common requirement to control the access remotely to these services. This may be to restrict some user groups or to avoid problems which can occur on heavy loads. When it comes to controlling access, it can be for a particular IP address range or for a particular domain. The level of control also can vary according to the entity that is being controlled. Some entity may be completely blocked while some other entity is allowed to access the service to a limited number of times per some time period. WSO2 Throttle module[1] can be used to fulfill this requirement. Throttling functionality comes as a feature in the WSO2 Carbon[2] platform. So you can use it as a quality of service feature in WSO2 Web Services Application Server[3] or as a mediator in WSO2 Enterprise Service Bus[4]. In addition to that, it can be used with Apache Axis2[5] as a normal module.

Levels of Application

WSO2 Throttling can be applied at different levels. When it is used as a quality of service feature for service hosting, it can be used at three different levels. When it is used with the WSO2 Enterprise Service Bus functionality, it is used as a mediator. So there are four levels of application for throttling:

Global level : Applies to all requests coming in to all hosted services
Service level : Applies to all requests coming in to all operations in the service
Operation level : Applies to all requests coming in to particular operation
Mediator level : Applies to all requests flowing through the mediator

How to Configure Throttling

In order to manage throttle configuration, there is a well defined throttling policy language. It is described in the next section of this article. But in order to make it easy to configure, throttling feature in WSO2 Carbon provides a user interface at service level and also at mediator level. But If you are familiar with throttling policy language, you can write policies and directly apply those. And also if you are using throttling module with Apache Axis2, you have to write the policies yourself.

Throttling Policy

Throttling policy language is defined by extending the WS-Policy framework[6]. As mentioned above, WSO2 throttling can be used with service hosting (global level, service level and operation level) or as a mediator in WSO2 Enterprise Service Bus functionality. Therefore, there are four different top level Throttling policy assertions defined as follows.

ModuleThrottleAssertion
ServiceThrottleAssertion
OperationThrottleAssertion
MediatorThrottleAssertion

Although top level assertions differ as above, internal configuration assertions inside the policy are the same for all the four types.

Example : A simple throttle policy

<wsp:Policy xmlns:wsp="https://schemas.xmlsoap.org/ws/2004/09/policy"
            xmlns:wsu="https://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
            xmlns:throttle="https://www.wso2.org/products/wso2commons/throttle"
            wsu:Id="WSO2MediatorThrottlingPolicy">
    <throttle:MediatorThrottleAssertion>
        <throttle:MaximumConcurrentAccess>10</throttle:MaximumConcurrentAccess>
        <wsp:Policy>
            <throttle:ID throttle:type="IP">other</throttle:ID>
            <wsp:Policy>
                <throttle:Control>
                    <wsp:Policy>
                        <throttle:MaximumCount>10</throttle:MaximumCount>
                        <throttle:UnitTime>800000</throttle:UnitTime>
                        <throttle:ProhibitTimePeriod wsp:Optional="true">10000</throttle:ProhibitTimePeriod>
                    </wsp:Policy>
                </throttle:Control>
            </wsp:Policy>
        </wsp:Policy>
    </throttle:MediatorThrottleAssertion>
</wsp:Policy>

Usage of internal configuration assertions

<throttle:MaximumConcurrentAccess>

This element can exist as a first level element of the Throttle Assertion. This assertion can be used to control the total number of requests that are served at any given moment. In other words, this is the maximum number of messages that can exist in between the inflow throttle handler and the outflow throttle handler. This is applied for the system without depending on any specific IP address or domain.

<throttle:ID throttle:type="**">**</throttle:ID>

Throttle “ID” assertion can be used to specify the party for which the access is throttled. “type” attribute can be used to specify whether this is an IP range or domain.

throttle:type=”IP” scenario :
ID can be a range of IP addresses or a single IP address.
Ex: 8.100.1.30 – 8.100.1.45, 8.100.1.60

throttle:type=”DOMAIN” scenario :
ID can be a domain.
Ex: *.wso2.org

“other” scenario :

When the value of the ID assertion is set to “other”, that means all other IP addresses and domains which are not configured in the other configurations.

<throttle:Allow/>

This assertion can exist in parallel with throttle:ID assertion as follows.

<wsp:Policy>
        <throttle:ID throttle:type="IP">10.100.1.130 - 10.100.1.143</throttle:ID>
        <wsp:Policy>
                 <throttle:Allow/>
        </wsp:Policy>
</wsp:Policy>

This means the specified throttle ID is allowed to access the system without any constraints.

<throttle:Deny/>

This assertion can exist in parallel with throttle:ID assertion as follows.

<wsp:Policy>
        <throttle:ID throttle:type="IP">10.100.1.130 - 10.100.1.143</throttle:ID>
        <wsp:Policy>
                 <throttle:Deny/>
        </wsp:Policy>
</wsp:Policy>

This means that access is completely denied for the specified throttle ID.

<throttle:Control>

This assertion can exist in parallel with throttle:ID assertion as follows.

<wsp:Policy>
         <throttle:ID throttle:type="IP">10.100.1.45 - 10.100.1.55</throttle:ID>
         <wsp:Policy>
                    <throttle:Control>
                             <wsp:Policy>
                                       <throttle:MaximumCount>10</throttle:MaximumCount>
                                       <throttle:UnitTime>10000</throttle:UnitTime>
                                       <throttle:ProhibitTimePeriod wsp:Optional="true">1000
                                       </throttle:ProhibitTimePeriod>
                             </wsp:Policy>
                    </throttle:Control>
         </wsp:Policy>
</wsp:Policy>

“Control” assertion communicates that access is controlled for the specified throttle ID. <wsp:Policy> assertion under the “Control” assertion is used to specify the configurations under which access is controlled.

<throttle:MaximumCount>**</throttle:MaximumCount>

This will be the maximum number of requests that are served within the time interval specified by the “UnitTime” parameter.

<throttle:UnitTime>**</throttle:UnitTime>

The time period in milliseconds during which only the above number of requests are allowed from this throttle ID.

<throttle:ProhibitTimePeriod wsp:Optional="true">**</throttle:ProhibitTimePeriod>

This is an optional assertion. If the maximum request count is achieved before the unit time, this is the period during which no more requests are allowed to go in. By setting this value, the unit time slot is altered.

Example Scenario :
MaximumCount(MRC) = 50, UnitTime(UT) = 50000, ProhibitTimePeriod(PTP) = 5000
If 50 requests are arrived within 35000ms (35s) in a particular time period, no more requests are taken in for another 5000ms (5s = PTP). Here, the UT is altered to 35000ms + 5000ms = 40000ms (40s)

More Complex Throttle Policy Example

<wsp:Policy wsu:Id="WSO2ServiceThrottlingPolicy"
            xmlns:wsp="https://schemas.xmlsoap.org/ws/2004/09/policy"
            xmlns:wsu="https://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
    <wsp:ExactlyOne>
        <wsp:All>
            <throttle:MediatorThrottleAssertion xmlns:throttle="https://www.wso2.org/products/wso2commons/throttle">
                <throttle:MaximumConcurrentAccess>100</throttle:MaximumConcurrentAccess>
                <wsp:Policy>
                    <throttle:ID throttle:type="IP">10.100.1.130 - 10.100.1.143</throttle:ID>
                    <wsp:Policy>
                        <throttle:Allow/>
                    </wsp:Policy>
                </wsp:Policy>
                <wsp:Policy>
                    <throttle:ID throttle:type="IP">10.100.1.20 - 10.100.1.40</throttle:ID>
                    <wsp:Policy>
                        <throttle:Deny/>
                    </wsp:Policy>
                </wsp:Policy>
                <wsp:Policy>
                    <throttle:ID throttle:type="IP">10.100.1.45 - 10.100.1.55</throttle:ID>
                    <wsp:Policy>
                        <throttle:Control>
                            <wsp:Policy>
                                <throttle:MaximumCount>10</throttle:MaximumCount>
                                <throttle:UnitTime>10000</throttle:UnitTime>
                                <throttle:ProhibitTimePeriod wsp:Optional="true">1000
                                </throttle:ProhibitTimePeriod>
                            </wsp:Policy>
                        </throttle:Control>
                    </wsp:Policy>
                </wsp:Policy>
                <wsp:Policy>
                    <throttle:ID throttle:type="DOMAIN">*.wso2.org</throttle:ID>
                    <wsp:Policy>
                        <throttle:Control>
                            <wsp:Policy>
                                <throttle:MaximumCount>30</throttle:MaximumCount>
                                <throttle:UnitTime>10000</throttle:UnitTime>
                                <throttle:ProhibitTimePeriod wsp:Optional="true">500
                                </throttle:ProhibitTimePeriod>
                            </wsp:Policy>
                        </throttle:Control>
                    </wsp:Policy>
                </wsp:Policy>
                <wsp:Policy>
                    <throttle:ID throttle:type="IP">other</throttle:ID>
                    <wsp:Policy>
                        <throttle:Control>
                            <wsp:Policy>
                                <throttle:MaximumCount>100</throttle:MaximumCount>
                                <throttle:UnitTime>10000</throttle:UnitTime>
                            </wsp:Policy>
                        </throttle:Control>
                    </wsp:Policy>
                </wsp:Policy>
            </throttle:ServiceThrottleAssertion>
        </wsp:All>
    </wsp:ExactlyOne>
</wsp:Policy>

In this example, MaximumConcurrentAccess is set to 100 and it applies to the system without depending on any IP address or domain.
There are 5 different throttles (throttle IDs) defined in this policy.

  • 10.100.1.130 – 10.100.1.143 : Access allowed for this IP range without any constraints.
  • 10.100.1.20 – 10.100.1.40 : Access completely denied for this IP range.
  • 10.100.1.45 – 10.100.1.55 : Access controlled for this IP range. Only 10 requests are allowed for a time period of 10 seconds. Prohibit period is 1 second.
  • *.wso2.org : Access controlled for this domain. Only 30 requests are allowed for a time period of 10 seconds. Prohibit period is 0.5 seconds.
  • other : Access controlled for all other IP addresses and domains as well. Only 100 requests are allowed for a time period of 10 seconds.

Throttling User Interface in Carbon

As mentioned above, if you are using throttling in Carbon (most probably in WSO2 Web Services Application Server and WSO2 Enterprise Service Bus), you have the luxury of configuring it using the UI without bothering to learn the throttling policy language. Following figure shows the user interface which is available in WSO2 Carbon.



This same UI is available at different places to facilitate the use of throttling at above mentioned four levels.

In WSO2 Web Services Application Server, throttling comes with the service hosting feature and it allows throttling at the following places.

Global level : Modules (under Manage) → List → Configure (wso2throttle) → Global throttling configuration page
Service level : Services (under Manage) → List → Service name link → Service Dashboard → Access Throttling → Service throttling configuration page
Operation level : Services (under Manage) → List → Service name link → Service Dashboard → Operations → Operation name link → Access Throttling → Operation throttling configuration page

In WSO2 Enterprise Service Bus, throttling comes as a mediator and can be found under advanced mediators.

Following table shows how the fields in this UI maps with the above explained policy configuration language.

Fields in the UI Corresponding Policy Element
Maximum Concurrent Accesses MaximumConcurrentAccess
Range ID
Type type attribute of ID assertion
Max Request Count MaximumCount
Unit Time (ms) UnitTime
Prohibit Time Period (ms) ProhibitTimePeriod
Access Deny, Allow or Control

In addition to that, you have a set of buttons which can be used to perform certain operations such as listed below.

Finish : Click Finish to submit the current configuration. When finished, your throttle configuration will be applied and the page will be redirected to the previous page.
Reset : Click Reset to load the last submitted configuration.
Default : Click Default to load the default throttle configuration.
Clear : Click Clear to clear all the text boxes in the UI.
Back : Click Back to go to the previous page.

Throttling Architecture

Throttling fits into message flows through handlers (Just like any other Apache Axis2 module[7]). There are handlers which fit into both inflow and outflow. When measuring the MaximumConcurrentAccess, it counts the total number of messages which exists between the throttle inflow handler and the throttle outflow handler. In all other cases, only the inflow handler is useful. Following diagram shows this high level architecture or throttling functionality.

Summary

In this article, first we discussed why throttling functionality is needed in production scenarios and how it is provided by the WSO2 Throttling module. Further, we had a look at how throttling can be configured through the throttling policy language and using the UI which is shipped with WSO2 Carbon. Finally, we learnt about the high level architecture of the Throttling module.

References

[1] Throttling svn location
[2] WSO2 Carbon
[3] WSO2 Web Services Application Server
[4] WSO2 Enterprise Service Bus
[5] Apache Axis2
[6] WS-Policy framework
[7] Apache Axis2 Modules

Author

Isuru Suriarachchi, Senior Software Engineer, WSO2, [email protected]

 

 

About Author

  • Isuru Suriarachchi
  • Technical Lead and Product Manager
  • WSO2 Inc.