Implementing libcurl based HTTP Transport in Apache Axis2/C
- By Dinesh Premalal
- 6 Jun, 2007
- Reads: 13004
Apache Axis2/C has a very extensible design and architecture. This article by Dinesh Premalal carries implementation details about libcurl based HTTP transport. Essentially, this document is targeted for an audience who has some exposure to Apache Axis2/C internals and for anyone who is seeking information about libcurl based transport implementation.
cURL is a command line tool used for URL manipulation. 'libcurl' is the API that facilitates using cURL inside our programs. There are many language bindings for libcurl, for instance PHP, PERL, Ruby, etc,. The advantage of giving HTTP transport level manipulation to such a library is precise and efficient handling of the protocol.
'libcurl' Integration to Apache Axis2/C
cURL transport is more focused on Apache Axis2/C client side request handling. It (libcurl) is responsible for sending SOAP messages through the HTTP protocol.We can provide the necessary information for cURL, such as HTTP headers, but we still need to have control over the protocol. You will understand how to address these issues while reading further.
Before building Axis2/C with libcurl, you need to make sure that libcurl is installed on your system.
Build with libcurl Support
./configure --enable-libcurl=yes [other options]
This option (--enable-libcurl) builds the system (Apache Axis2/C) with AXIS2_LIBCURL_ENABLED defined. The Apache Axis2/C HTTP transport sender will decide what to use based on this definition.
Set the following options in build/configure.in file
LIBCURL_BIN_DIR = "path to libcurl bin directory"
typedef struct axis2_libcurl
unsigned int size;
const axis2_env_t *env;
libcurl based HTTP transport implementation of Axis2/C is done by using the libcurl's 'easy' interface. The axis2_libcurl_t structure's 'memory' element is used for holding a response, and the 'size' element is used to hold its (memory element) size.
curl_easy_setopt () is used to set various options and
curl_easy_perform () is used to perform the operation (send request and get response).
libcurl Options Used
HTTP Header Manipulation
Before you do anything with cURL, it's essential to initialize it.
handler = curl_easy_init ();
The initialized easy handler is used to set various options and perform the operation.
In libcurl, the user can manipulate the HTTP header through a structure called curl_slist .
struct curl_slist *headers = NULL;
If you need to add or remove HTTP headers, you can use
curl_slist_append. The addition of headers can be done as follows.
headers = curl_slist_append (headers, "User-Agent:Axis2/C");
By default, cURL sends some HTTP headers but Axis2/C doesn't have any special use of them. You can remove them as follows,
headers = curl_slist_append (headers, "Accept:");
headers = curl_slist_append (headers, "Expect:");
Once you have the necessary headers in hand, you need to set them to easy handler using
curl_easy_setopt (handler, CURLOPT_HTTPHEADER, headers);
HTTP Headers Used
The HOST HTTP header is one of the headers that libcurl uses. We also need to have this in our request. Therefore, we keep this header unchanged.
For Axis2/C, the user agent should be "Axis2/C". We set this header in a single line.
headers = curl_slist_append (headers,"User-Agent:Axis2/C");
The buffer length of the SOAP is used as a content length. You can directly obtain the buffer length from the XML writer.
The content type differs based on the SOAP or REST request, SOAP1.1 or SOAP 1.2, MTOM enabled or disabled, and the char set encoding. Accordingly, we create the 'content-type' header string and append it to the 'curl_slist' that we have already created.
SOAP Action is a mandatory header for SOAP 1.1 requests. After determining whether the request is SOAP 1.1, you can obtain the SOAP action from the message context. (The message context is a context that holds certain properties about a message.)
HTTP Body/Content Management
The SOAP request that we got from the XML parser has to be handed over to the HTTP request. This is done by using CURLOPT_POSTFIELDS and the size of the buffer is set as CURLOPT_POSTFIELDSIZE.
curl_easy_setopt (handler, CURLOPT_POSTFIELDSIZE, buffer_size);
curl_easy_setopt (handler, CURLOPT_POSTFIELDS, buffer);
curl_easy_setopt (handler, CURLOPT_URL, str_url);
The URL of the endpoint is set as above.
When handling HTTP GET, it is essential to append some data fields to a URL and use 'url encode' for a safe transfer. Still libcurl does not handle it as a part of its job. Therefore, we need to handle that scenario using a third party library. Apache Axis2/C uses an internal function to send a REST request through HTTP GET.
Since we have to read the response data to a buffer, we must register a callback in libcurl in the following format.
size_t function( void *ptr, size_t size, size_t nmemb, void *stream);
In Axis2/C, we use a callback as follows.
"ptr" is the data pointer given by the cURL. The size of that ptr is a product of the "size" to "nmemb". "data" is a structure that holds incoming data according to the cURL. This function is untouched. Register this function and the stream that is provided to hold data.curl_easy_setopt (handler,CURLOPT_WRITEFUNCTION, axis2_libcurl_write_memory_callback);
curl_easy_setopt (handler, CURLOPT_WRITEDATA, data);
If we did not provide this callback, libcurl will write to stdout.
As with the WRITEFUNCTION, if we intend to receive HTTP headers, we must register a callback in the following format,
size_t function( void *ptr, size_t size, size_t nmemb, void *stream);
using the CURLOPT_HEADERFUNCTION, CURLOPT_WRITEHEADER.
curl_easy_setopt (handler, CURLOPT_VERBOSE, 1);
If you are doing any development using libcurl, you should be aware of this function. When CURLOPT_VERBOSE is set to 1, it prints details about the operations it carries out.
Performing various functions with libcurl in Axis2/C is all about setting the applicable options. After setting the options, you can perform the sending task.
There has to be a mechanism to read data on request. Although we have response data on hand, in order to have a smooth integration, the libcurl stream implementation is required. Since Apache Axis2/C has an extensible architecture, there is not much work to be done to plug in the stream implementation.
To implement the libcurl stream, the interface available in axis2_stream.h is used. After the implementation, the stream is set to the message context as a property named AXIS2_TRANSPORT_IN. Later, this stream is set to a callback, which is used to build the response SOAP message.
Handling HTTP protocols through a library like 'libcurl' has the advantage of fine tuned and precise communication. It is a widely used and well tested transport library. It has many HTTP protocol handling features like HTTP-Chunking and IPV6. Apache Axis2/C will be able to gain these advantages by using this implementation.
Dinesh Premalal, Senior Software Engineer, WSO2 Inc. dinesh at wso2 dot com