WSO2Con 2013 CFP Banner

Apache Sandesha2 Configuration Framework

Discuss this article on Stack Overflow
By Chamikara Jayalath
  • 5 Jul, 2006
  • Level: 
  • Reads: 4760
Apache Sandesha2 is a reliability framework for Apache Axis2. By engaging Sandesha2 module to your Axis2 based services, you can make them reliability aware and clients will be able to invoke them in a reliable manner. Reliability and performance do not always go together. But by configuring Sandesha2 module correctly, you will be able to get the best of both worlds. In this article WSO2's Chamikara Jayalath introduces you to the configuration framework of Apache Sandesha2 through which you can achieve the above goal.
Chamikara Jayalath
Software Engineer
WSO2 Inc.

Introduction

Apache Sandesha2 (version 1.0) has chosen WS-Policy as its configuration language. So almost all of the configurations of Sandesha2 will be within a policy element. At fist glance one may doubt the necessity of following this standard. But following WS-Policy rules could help a lot when the configuration details become more complex. This is also very helpful when users need to specify certain sets of configurations as the final decision can be made using a simple policy merge. Sandesha2 underneath uses the Apache Neethi framework to manipulate WS-Policies.

Policy Levels

Policies can be fed into Sandesha2 framework in various levels as given below.
  1. Module level
  2. Service level
  3. Operation level
Module level policies give the default configuration values of a Sandesha2 module. These policies are available in the module configuration file, namely module.xml. Service level policies affect the way Sandesha2 module work for a certain Apache Axis2 service. These are mentioned in the service configuration file of your Axis2 service archive, namely services.xml. Operation level policies will be mentioned within a 'operation' element of a service configuration file. The service level policies will override the values given in the module.xml when it comes to that specific service. Similarly operation level policies will override any policy values mentioned in the service or module levels. This is given in the diagram below.
Internally Sandesha2 achieves this by using the Context Hierarchy feature of Axis2. All the policy values of Sandesha2 are mentioned in a Sandesha2PropertyBean object. When the module initializes, a bean instance is created using the default values mentioned in the module configuration file. This is stored as a property in the AxisConfiguration object which will be available to the whole axis2 system. When Sandesha2 is engaged to a certain service, any policy values of that AxisService are used to create a service level Sandesha2PropertyBean object. The values that are not present in this level are obtained from the parent level of the Context Hierarchy which is the AxisConfiguration. This way Sandesha2 could show a policy overriding behavior. Similarly any policies available in a operation level will again extract the absent policy values from the parent level which could be the AxisService or AxisConfiguration.

Current Sandesha2 Policies

In this section we will go into a detailed analysis of each Sandesha2 policy. You will be given the reason to use each of these policies and the information to set the values of these policies in the most affective manner.

AcknowledgementInterval

This policy is about the way Sandesha2 should send acknowledgments. Acknowledgments are an essential part of any Reliability protocol and configuring correctly sending them is a key point to the performance of your Reliable Mesasing system. Acknowledgment messages could be sent as standalone or piggybacked to a different message. When your have given a real endpoint for your ack endpoint (I.e. When it is not equal to the anonymous URI) Sandesha2 will wait the time given by this policy value before sending the acknowledgment message as an standalone one. If any other message to the same endpoint come within this time interval, the acknowledgment will be piggybacked. If your service takes time to send response messages, waiting this time period will be a waste. In such a scenario you can set the AcknowledgementInterval to a lower value and make your system more efficient.

RetransmissionInterval and ExponentialBackoff

Reliable Messaging frameworks usually use message retransmission techniques to guarantee the delivery of messages. This is true for Web services as well. If an acknowledgment is not received within a given time interval a message will be retransmitted. Sandesha2 decides this interval based on two policies. If the boolean 'ExponentialBackoff' property is set to false, this will be equal to the millisecond value given by the 'RetransmissionInterval' property. In this scenario the interval between two successive retransmission attempts will be constant. But if the boolean property is set to 'true' the interval between two successive retransmission attempts will not be the same. In every retry this interval will be multiplied by two. For exaple, if the 'RetransmissionInterval' is set to 3000 and if ExponentialBackoff is 'true', the time interval between successive retransmission intervals (in milliseconds) will be 3000, 6000, 12000, 24000 etc. You have to set the retransmission interval based on the ability of your server to handle requests. If your endpoint can only handle a smaller number of requests per second, it is safer to keep the ExponentialBackOff property to 'true' and keep the RetransmissionInterval value reasonably lower. This will make the server performance high while saving it from a burst of acknowledgment messages.

MaximumRetransmissionCount, InactivityTimeout and InactivityTimeoutMeasure

These three properties are used to decide when Sandesha2 should give up working on a message sequence. First thing to note is the MaximumRetransmissionInterval and the InactivityTimeout, two policies which could be used to achieve the same target. MaximumRetransmissionCount tells the Sandesha2 system to give up a sequence if the message delivery fails after a certain number of retries. InactivityTimeout tells it to give up only if the other endpoint does not reply in the given timeout interval. You will most probably need only one of these policies. Keep the value of the other policy at '-1' to avoid it from affecting the system. The InactivityTimeoutMeasure policy is used to allow the users to enter the inactivity timeout value in a more user friendly manner. The inactivity timeout can go up to a number of days. So if you can only enter the value in seconds (which is the default) you will have to do a calculation and enter quite large value to set your InactivityTimeout to a number of days. Sandesha2 makes this easier by introducing this new policy called InactivityTimeoutMeasure. You can set its value to seconds, minutes, hours or days. This will be considered as the measure for the value you have entered in the InactivityTimeout policy.

InvokeInOrder

In-Order invocation is one of the delivery assurances provided by Sandesha2. The idea is to invoke the messages according to the order they were numbered by the RM sender. Even though this is a nice and a useful feature, this could have a considerable affect on the performance. Specially if most of your messages arrive at the destination out of order. Suppose the RM Sender sends five messages numbered from one to five to the RM Destination. If the first message that arrives at the RM destination as message number four it will not be invoked until the RM destination receive the previous three messages (numbered from one to three). Consider this a scenario where RM Sender can send only one message at a time and has to wait for an acknowledgment before sending another message, in which case the two endpoints will be stuck in a deadlock. As you can see, even though In-Order invocation is a good feature, there are times where you have to disable it. This will give your system a higher performance. Remember that even if you disable the in-order invocation, Sandesha2 will still guarantee exactly-once delivery assurance. I.e., it will make sure that it invokes only once, a message of a given sequence with a certain message number. Any future messages that belong to the same sequence and have the same message number will be discarded. Currently InvokeInOrder policy is non-overridable. I.e., the value mentioned in the module.xml will not be overridden by mentioning a different value in the services.xml. Therefore, you cannot ask Sandesha2 to invoke only a set of services In-Order. It has to be a system wide policy that affect all the services.

StorageManagers

StorageManager concept is one of the most important principles behind Sandesha2. This is the feature that allows rest of the code in Sandesha2 to be independent of the underlying storage mechanism. Sandesha2 policies allow users to set two storage managers. One for a in-memory implementation and other for a permanent storage implementation. What you have to mention is the full name of the Storage Manager implementation class including the package name. Again, this is a non-overridable property. I.e., the two Storage Manager class names you mention in the module.xml will not be overridden by a policy set in the services.xml. So how do you pick the affective storage manager class. How do you tell the Sandesha2 system to work on the permanent or in-memory storage manager? This is easy. You simply have to add a property to your axis2 configuration file or Axis2.xml. If you want to tell Sandesha2 to work on the permanent storage manager add following to the axis2.xml:
<parameter name="Sandesha2StorageManager"
locked="false">inmemory</parameter>
If you want it to work on the permanent Storage Manager. Change the property value as follows:
<parameter name="Sandesha2StorageManager"
locked="false">permanent</parameter>
If this property is not available, Sandesha2 will pick the default, which is the in-memory implementation.

Working with Policy Values

Viewing and Changing Module Level Policies

When you obtain the Sandesha2 module from a Sandesha2 distribution or by building Sandesha2 from source, it will always come up with a default set of policies. Extract your Sandesha2 module and view the module.xml file which comes under META-INF directory. You will find a set of policies similar to following:
<wsp:Policy >
<wsrm:AcknowledgementInterval>3000</wsrm:AcknowledgementInterval>
<wsrm:RetransmissionInterval>6000</wsrm:RetransmissionInterval>
<wsrm:MaximumRetransmissionCount>10</wsrm:MaximumRetransmissionCount>
<wsrm:ExponentialBackoff>false</wsrm:ExponentialBackoff>
<wsrm:InactivityTimeout>60</wsrm:InactivityTimeout>
<wsrm:InactivityTimeoutMeasure>seconds</wsrm:InactivityTimeoutMeasure>
<wsrm:InvokeInOrder>true</wsrm:InvokeInOrder>
<wsrm:MessageTypesToDrop>none</wsrm:MessageTypesToDrop>
<wsrm:StorageManagers>
<wsrm:InMemoryStorageManager>
org.apache.sandesha2.storage.inmemory.InMemoryStorageManager
</wsrm:InMemoryStorageManager>
<wsrm:PermanentStorageManager>
org.apache.sandesha2.storage.inmemory.InMemoryStorageManager
</wsrm:PermanentStorageManager>
</wsrm:StorageManagers>
</wsp:Policy>
These are the default set of policies of Sandesha2. If you want the change the default values you need to follow the given steps:
  1. Copy the Sandesha2.mar to a temporary directory.
  2. Extract the contents by issuing the 'jar' command.
  3. jar -xvf Sandesha2.mar
  4. Change the policy value of the module.xml file as necessary.
  5. Delete the Sandesha2.mar file.
  6. Re-compact the Sandesha2.mar using the 'jar' tool.
  7. jar -cvf Sandesha2.mar *

Adding Higher Level Policies

But a better way may be to keep the default values as they are and change the policy values for a specific service or operation. In Axis2 you deploy services as a '.aar' archives (read Apache Axis2 User Guide to learn more about this). These archives should contain a services.xml file within a META-INF directory. You can include a set of Sandesha2 policies in this services.xml file. These policies will override the default policies given in the services.xml for this service only. An example for a services.xml with service level policies is given below:
<service name="RMSampleService">
<parameter name="ServiceClass"></parameter>
<operation name="ping" mep="http://www.w3.org/2004/08/wsdl/in-only"></operation>
<operation name="echoString"></operation>

<supported-policy-namespaces
namespaces="http://ws.apache.org/sandesha2/policy" />
<wsp:Policy xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy"
xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/
oasis-200401-wss-wssecurity-utility-1.0.xsd"
xmlns:wsrm="http://ws.apache.org/sandesha2/policy" wsu:Id="RMPolicy">

<wsrm:AcknowledgementInterval>3500</wsrm:AcknowledgementInterval>
</wsp:Policy>
</service>
The great thing about this is the ability the override only a set of policies you need while letting Sandesha2 to pick other policies from the default values given in the module.xml. This is what has been done in the above example where AcknowledgementInterval policy will be picked from the servicex.xml while others get picked from the module.xml. As explained earlier you can go one step further by adding policies under the 'operation' element of the services.xml as well. These policies become affective once it comes to that operation, while others are picked from the service/module levels respectively. There are some policies of Sandesha2 that cannot be overridden. This is such mainly due to the limitations in the current implementation (Apache Sandesha2 1.0) and will most probably be overridable in the future. Currently there are two non-overridable policies.
  1. InvokeInOrder
  2. StorageManagers
What will happen if you do not mention any policies in any of these module, service or operation levels? Well, in that case, Sandesha2 will derive a default set of policies from its internal Sandesha2Constants class. Most of the time these would have values similar to the ones given in the default module.xml.

Conclusion

The key to get the best out of any software component is configuring it correctly. Sandesha2 is not exception. Even though your web service can be reliability aware by simply engaging this module, there is more you should to to make it work in the most efficient manner.

Resources

February 2004, WS-ReliableMessaging May 2006, Axis2 User Guide May 2006, Sandesha2 User Guide

Author

Chamikara Jayalath, Software Engineer, WSO2 Inc. chamikara@wso2.com