Most of the time, Web services communicate using SOAP, but occasionally users need to transmit binary attachments such as images, videos or drawings along with Web service requests. In this article, Manjula Pieris, software engineer at WSO2, explains how Axis2/C sends and receives these attachments.
Table of Contents
Attachment Sending Mechanisms Supported by Axis2/C
There are two major types of attachment sending mechanisms with SOAP:
Keep the attachment inside the SOAP message
Keep the attachment outside the SOAP message
The Attachment Inside the Message
Since SOAP is XML, it is a text format, while most of the attachments are in binary format. This method converts the attachment into a text format using an encoding mechanism such as Base64Encoding. After converting the attachment, data can be inserted as XML text. The attachment is kept inside the SOAP message while transmitting, which is useful when you need to sign or encrypt the attachment. Since it is text, XML signature or XML encryption can be applied as normal.
There are inherent disadvantages to this technique. The SOAP message can become very large, because the encoded size is much larger than the normal binary (1.33x with base64, 2x with HexBinary) and the processing overhead (encoding and decoding).
A sample message from Axis2/C:
POST /axis2/services/mtom HTTP/1.1
The Attachment Outside the Message
In this method, the attachment is kept outside the SOAP message, with the SOAP message containing a reference to these external parts. The attachments are not converted to any other text format, which prevents unnecessary bloating of data and waste of processing power. This technique is called SWA (SOAP With Attachments). This creates two data models within the same message, which prohibits some concurrent SOAP level message operations such as signing and encryption. Axis2/C does not fully support SWA.
MTOM (Message Transmission Optimization Mechanism) is a combination of these two techniques. The wire level of the MTOM message is the same as the SWA message. But MTOM has standardized the SWA referencing mechanism using an XML element called XOP:Include. This is defined in XML-Binary Optimized Packaging. Even though the attachment data is attached externally to the message, it is referenced using a standard technique. Hence any SOAP or XML processing software can consider this to be within the XML infoset. Whenever these elements are encountered, the referenced portions can be converted to a text format and can be thought of as an XML infoset. The advantage is that the aforementioned operations such as XML encryption and Signature can be applied. Because the wire format is the same as SWA, the message will not become unnecessary large while transmitting. In addition, MTOM is backward compatible with SWA, so any SWA supporting agent can process MTOM messages.
A sample MTOM message from Axis2/C:
POST /axis2/services/mtom HTTP/1.1
Axis2/C supports sending attachments as base64 encoded and MTOM. Axis2/C's Axiom object model is equipped for handling attachments. Axiom enhances its XML handling capability with attachments using an API called axiom_data_handler. axiom_text API is enhanced using axiom_data_handler API to accept attachments as axiom_text. With the recent improvements to Axiom, Axis2/C can now handle large attachments while consuming very little memory footprint (configurable by the user). Users can easily achieve this by just setting some parameters in the axis2.xml. Please refer to the section Receiving Attachments in this article for this feature.
The steps to follow when sending an attachment using an Axis2/C client are:
- Enable or disable MTOM
- Create the axiom_data_handler
- Create axiom_text using the data handler
- Set the optimized flag
The following describes the client code for sending attachments using MTOM.
First set the MTOM flag in axis2_options_t using the following function:
axis2_options_set_enable_mtom(options, env, AXIS2_TRUE);
Then create the payload:
axiom_node_t *mtom_om_node = NULL;
axiom_element_t *mtom_om_ele = NULL;
axiom_node_t *attachment_om_node = NULL;
axiom_element_t *attachment_om_ele = NULL;
axiom_node_t *data_om_node = NULL;
axiom_text_t *data_text = NULL;
axiom_namespace_t *ns1 = NULL;
axis2_char_t *om_str = NULL;
axiom_data_handler_t *data_handler = NULL;
ns1 = axiom_namespace_create(env, "http://ws.apache.org/axis2/c/samples/mtom", "ns1");
mtom_om_ele = axiom_element_create(env, NULL, "mtomSample", ns1, &mtom_om_node);
attachment_om_ele = axiom_element_create(env, mtom_om_node, "attachment", ns1, &attachment_om_node);
data_handler = axiom_data_handler_create(env, file_name, "image/jpeg");
data_text = axiom_text_create_with_data_handler(env, attachment_om_node,data_handler,&data_om_node);
axiom_text_set_optimize(data_text, env, AXIS2_TRUE);
To send the attachment as base64 encoded text, simply set optimize flag to false.
axiom_text_set_optimize(data_text, env, AXIS2_FALSE);
Different Types of Sources to Create Attachments
Axis2/C data_handler interface is very flexible. The user can set a file, a buffer or a callback as the attachment's data source. The following code segment shows how to use these different facilities.
To create an axiom_data_handler using an image file:
data_handler = axiom_data_handler_create(env, "image.jpg", image/jpeg);
To create using a buffer:
data_handler = axiom_data_handler_create(env, NULL, NULL);
axiom_data_handler_set_binary_data(data_handler, env, buffer, buffer_length);
To create using a callback:
data_handler = axiom_data_handler_create(env, NULL, NULL);
axiom_data_handler_set_data_handler_type(data_handler, env, AXIOM_DATA_HANDLER_TYPE_CALLBACK);
axiom_data_handler_set_user_param(data_handler, env, void *(user_param));
In the callback case, the user needs to set the axiom_data_handler_type_t as AXIOM_DATA_HANDLER_TYPE_CALLBACK. The path of the callback should be specified in the axis2.xml <MTOMSendingCallback> parameter. The user_param can be a file_name if the callback is going to load data from a file. If the attachment is loaded from a database, this value can be a key to load data. The callback should know how to handle this value. The callback should be implemented using axiom_mtom_sending_callback.h.
Axis2/C can receive very large attachments, and it is not dependent on the available memory.
Users can specify that large attachments should be cached either to a file or to any storage. In order to enable caching, the user should set either "attachmentDir" or "MTOMCachingCallback" parameters in the axis2.xml. If both are set, the callback will be used; iff nothing is set attachment will reside in memory.
Following is an example of specifying the attachmentDir.
<parameter name="attachmentDIR" locked="false">/path/to/the/dir/</parameter>
The attachments will be saved in the specified directory using the attachment content id as the file name.
The callback should be implemented using axiom_mtom_caching_callback.h. The following parameter will enable the caching callback:
<parameter name="MTOMCachingCallback" locked="false">/path/to/the/attachment_caching_callback</parameter>
A sample callback implementation can be found here.
Axis2/C allows you to set the caching threshold. The default is 1MB. As an example, to cache attachments which are greater than 10MB in size, the user needs to add the following directive in axis2.xml:
<parameter name="MTOMBufferSize" locked="false">10</parameter>
This will give users the control to use the availbale memory even with larger attachments.
When receivng attachments, first check the data_handler. If you can get a data_handler from the om_text node that means, then the attachment is optimized. The following code will retreive the data_handler:
axiom_data_handler_t *data_handler = NULL;
axiom_text_t *bin_text = NULL;
data_handler = axiom_text_get_data_handler(bin_text, env);
After gettting the data_handler, you must ascertain whether it is cached or not. If the caching was done to a file enabling "attachmentDir", then the attachment can be found calling axiom_data_handler_get_file_name() function. Alternatively, if the caching was done through a callback, the user should know how to handle it because the callback was implemented according to the user requirements. The following code segment illustrates how to handle the caching case:
data_handler_type = axiom_data_handler_get_data_handler_type(data_handler, env);
if(data_handler_type == AXIOM_DATA_HANDLER_TYPE_CALLBACK)
/* User should handle this logic */
else if(data_handler_type == AXIOM_DATA_HANDLER_TYPE_FILE)
axis2_char_t *file_name = NULL;
file_name = axiom_data_handler_get_file_name(data_handler, env);
/* User can do whatever */
For the non-cached case the attachment will be in memory. The buffer and length can be accessed using functions in the axiom_data_handler.h.
if (!axiom_data_handler_get_cached(data_handler, env))
axis2_byte_t *buffer = NULL;
int buff_len = 0;
input_buff = axiom_data_handler_get_input_stream(data_handler, env);
buff_len = axiom_data_handler_get_input_stream_len(data_handler, env);
/* User can do whatever with the buffer */
Axis2/C MTOM Callbacks
This section describes Axis2/C MTOM sending and receiving callbacks.
With HTTP, Axis2/C sends MTOM messages as http chunks. The user can determine what portion of the binary attachment will be loaded before sending. Axis2/C will call the functions in the user-implemented callback in order to load this portion before writing to the wire. This allows users to carefully use the memory available, even when sending very large attachments. The sending side callback should be implemented according to the axiom_mtom_sending_callback.h.
axiom_mtom_sending_callback.h defines a structure called axiom_mtom_sending_callback_ops. This structure contains four function pointers. The callback implementer should implement these functions and assign them to the function pointers in his callback. In addition, this should provide a buffer to load data from the data store and should define MAX_BUFFER_SIZE directive to determine the maximum size of the buffer.
Each function should pass a pointer to an axiom_mtom_sending_callback_t struct. Because Axis2/C calls this function using a macro, this struct contains a pointer to the axiom_mtom_sending_callback_ops struct.
void* (AXIS2_CALL* init_handler)(axiom_mtom_sending_callback_t *mtom_sending_callback, const axutil_env_t* env, void *user_param);
The init function will take a void pointer to a user_param, which is meaningful inside the callback function implementation. For example, if the user is loading data from a file this may be a pointer to the file name. If it is from a database, this may be a unique ID to retrieve data. This function will open the data storage, create the buffer to load the data and return these details as a void pointer.
int (AXIS2_CALL* load_data)(axiom_mtom_sending_callback_t *mtom_sending_callback, const axutil_env_t* env, void *handler, axis2_char_t **buffer);
Axis2/C will pass the handler created in the init call with a double pointer to a buffer and call this function in a loop. Inside this function the data should be read to the buffer created in the init call. Finally, this buffer should be assigned to the double pointer passed. The size of the data read into the buffer will be returned. If there is no more data, zero should be returned.
axis2_status_t (AXIS2_CALL* close_handler)(axiom_mtom_sending_callback_t *mtom_sending_callback, const axutil_env_t* env, void *handler);
This function will close the handler and release all the resources created from the callback. For example, this would free the buffer created in the init call. In a file case this will close the file, and in a database case this will close the connection.
axis2_status_t (AXIS2_CALL* free)(axiom_mtom_sending_callback_t *mtom_sending_callback, const axutil_env_t* env);
This will free all the callback structs.
A sample implementation can be found here. This sample uses a file as the storage.
This is the callback used to cache the attachment when receiving. Axis2/C will call the functions in this callback when it is parsing the MTOM message. After the message is parsed, the attachment will be stored where the callback is supposed to store. This callback should be implemented according to axiom_mtom_caching_callback.h. After initializing a handler, Axis2/C will call the caching function repeatedly to store data. It is callback implementer's responsibility to keep this data in memory or in storage.
axiom_mtom_caching_callback.h defines a struct called axiom_mtom_caching_callback_ops. This structure contains four function pointers. The callback implementer should implement these functions and assign them to these function pointers in his callback. Each function should pass a pointer to an axiom_mtom_caching_callback_t struct. Because Axis2/C calls these functions using a macro, this struct contains a pointer to the axiom_mtom_caching_callback_ops struct.
void* (AXIS2_CALL* init_handler)(axiom_mtom_caching_callback_t *mtom_caching_callback, const axutil_env_t* env, axis2_char_t *key);
This function will initiate the storage. This key will be the content-id of the particular attachment. Since this is unique to each attachment, the callback implementer can uniquely store the attachment. This function will init the storage and return a handler wrapped as a void pointer to Axis2/C. Axis2/C will then use this handler repeatedly to call the cache function.
axis2_status_t (AXIS2_CALL* cache)(axiom_mtom_caching_callback_t *mtom_caching_callback, const axutil_env_t* env, axis2_char_t *data, int length, void *handler);
This function should store large amounts of data pointed by the argument axis2_char_t *data using the handler. Implementation of this function should be in such a manner that the repeated calls will append the data to the storage.
axis2_status_t (AXIS2_CALL*close_handler)(axiom_mtom_caching_callback_t *mtom_caching_callback, const axutil_env_t* env, void *handler);
This function will close the handler and will release all the resources created from the callback.
axis2_status_t (AXIS2_CALL* free)(axiom_mtom_caching_callback_t *mtom_caching_callback, const axutil_env_t* env);
This function will free the callback structures. A sample implementation can be found here. This sample uses a file as the storage.
A sample client which sends and receives attachments can be found here, and a sample server which uses theses callbacks can be found here.
Axis2/C has a very flexible API to send and receive attachments. It supports sending attachments as base64 encoded and sending attachments as binary. Axis2/C has a very rich API for MTOM, which is the most widely used and interoperable mechanism for sending and receiving attachments. Axis2/C can handle very large attachments while using a small memory footprint. The available memory can be optimized by using the configurations. Axis2/C also provides callback mechanisms to load attachments from different sources as well as storing them in different storages.
Manjula Pieris is a Software Engineer at WSO2. manjula at wso2 dot com