WSO2 Venus

Managing artifacts of a service oriented solution is one of the most important features expected from any SOA middleware platform. In a typical service oriented project, SOA artifacts are subjected to move through multiple phases. Usually, separate environments are maintained for the activities associated with those phases.
e.g:-

Development environment

Service development and system integration tasks are carried out in a separate physical environment. Depending on the requirements, there can be multiple SOA middleware solutions used in development environment to facilitate service development, integration and deployment processes.Once the development tasks are completed, SOA artifacts are transferred into QA environment for QA verification.

QA environment

Solution testing is done in an independent environment which is usually identical to the production settings. Various functional and non-functional tests are performed in QA environment. Upon successful QA verification, the artifacts are moved to production or staging environment

When the service oriented solution becomes complex and there are large number of SOA artifacts, automated artifact governance mechanism is required to ensure smooth transition between various environments. In this post, I will take you through a simplified SOA artifact governance process using WSO2 SOA middleware stack. We will explore a use case similar to the following.

Separate development and QA environments are used to maintain the SOA artifacts. For example, solution developers implement various configuration artifacts in ESB development node. After completing those development tasks, solution developers move those artifacts in to QA environment which allows the testers to carryout QA activities in an independent environment. The moving of artifacts will be done automatically using the features provided by central SOA governance solution.
In this example, we will use WSO2 ESB as the enterprise service bus middleware, WSO2 Governance Registry as the SOA governance solution.

Lets go through each of the steps in detail. We need to have 2 WSO2 ESB instances and 1 WSO2 G-reg instance which runs on mySQL. All these instances will be deployed in single host.

Setting up G-reg and ESB products
Step 1
Download the latest versions of WSO2 ESB and WSO2 G-reg from here. First we will configure WSO2 G-reg instance. Extract wso2greg-4.x.x.zip into your file system. We will refer to the extracted location as GREG_HOME.
By default, WSO2 products run on the file based embedded H2 database. Since we need two ESB instances connect to the database used by G-reg instance, we will configure G-reg to run on a mySQL database. Enter the following commands to create a database and grant permission to a user.

mysql> create database greg_db;
Query OK, 1 row affected (0.00 sec)

mysql> use greg_db;
Database changed
mysql> grant all on greg_db.* to regadmin@localhost identified by "regadmin";
Query OK, 0 rows affected (0.05 sec)

Step 2
Now, we need to change the default database configuration in G-reg instance through registry.xml configuration file. Open GREG_HOME/repository/conf/registry.xml and update dbConfig element as shown below.

<dbConfig name="wso2registry">
        <url>jdbc:mysql://localhost:3306/greg_db</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>50</maxActive>
        <maxWait>60000</maxWait>
        <minIdle>5</minIdle>
    </dbConfig>

Copy mysql jdbc driver to GREG_HOME/repository/components/lib and start G-reg server by running wso2server.sh

Step 3
We have completed the configurations of WSO2 Governance Registry instance. Lets move forward with setting up ESB instance in development environment.

Extract the downloaded wso2esb-4.x.x.zip into a directory in your file system. We will refer to this directory as ESBDEV_HOME. Since we are running all product instances in a single machine, we need to start them up in different ports. Therefore, change port offset parameter in ESBDEV_HOME/repository/conf/carbon.xml as follows.

<Offset>1</Offset>

Step 4
As shown in the above diagram, we will mount /_system/config collection, where the ESB artifacts are stored in to /_system/dev collection of G-reg instance. ESBDEV_HOME/repository/conf/registry.xml is used to configure that.

<dbConfig name="configgovregistry">
        <url>jdbc:mysql://localhost:3306/greg_db</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>50</maxActive>
        <maxWait>60000</maxWait>
        <minIdle>5</minIdle>
</dbConfig>
<remoteInstance url="https://localhost:9443/registry">
        <id>configgov</id>
        <dbConfig>configgovregistry</dbConfig>
        <readOnly>false</readOnly>
        <enableCache>true</enableCache>
        <registryRoot>/</registryRoot>
</remoteInstance>

<mount path="/_system/config" overwrite="true">
        <instanceId>configgov</instanceId>
        <targetPath>/_system/dev</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
        <instanceId>configgov</instanceId>
<targetPath>/_system/governance</targetPath>
    </mount>

Start ESB dev instance. If you log in to management console of the ESB instance (https://localhost:9444/carbon) and access registry browser, you will notice the config and governance collections are mounted (See the arrow icons) to the remote G-reg database.

Step 5

Repeat step 3 and 4 with another copy of wso2esb-4.x.x.zip. Lets call it ESBQA instance. (The root directory of ESBQA instance will be referred to as ESBQA_HOME). Make sure to specify a different offset value in carbon.xml. In this instance, we will mount /_system/config collection to /_system/qa collection of G-reg instance. Update registry.xml of ESBQA node as follows.

<dbConfig name="configgovregistry">
        <url>jdbc:mysql://localhost:3306/greg_db</url>
        <userName>regadmin</userName>
        <password>regadmin</password>
        <driverName>com.mysql.jdbc.Driver</driverName>
        <maxActive>50</maxActive>
        <maxWait>60000</maxWait>
        <minIdle>5</minIdle>
</dbConfig>
<remoteInstance url="https://localhost:9443/registry">
        <id>configgov</id>
        <dbConfig>configgovregistry</dbConfig>
        <readOnly>false</readOnly>
        <enableCache>true</enableCache>
        <registryRoot>/</registryRoot>
</remoteInstance>

<mount path="/_system/config" overwrite="true">
        <instanceId>configgov</instanceId>
        <targetPath>/_system/qa</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
        <instanceId>configgov</instanceId>
<targetPath>/_system/governance</targetPath>
    </mount>

Start the server. We are done with the product configurations. However, we have not done any artifact governance configuration yet. In the next steps, we will look into moving artifacts between the two ESB instances.

Using WSO2 Governance Registry to move artifacts between environments
As I explained at the beginning, we use central governance registry instance to manage the artifacts produced by each of DEV and QA environments. The ESB artifacts produced by development environment are stored under /_system/dev collection of governance registry where as the artifacts used in QA environment are stored in /_system/qa collection. When the development tasks are compelte and ready for QA, we can manually copy the artifacts into the relevant locations and configure the QA environment. Obviously it will be a painful task to copy large number of artifacts into various locations of ESB QA environment by hand. Therefore, we need some kind of automated artifact copying mechanism.
WSO2 Governance Registry provides us with registry extension features to extend the core functionality of G-reg to use in these types of situations. The lifecycle management feature is such a useful extension provided by WSO2 Governance Registry which can be used to manage life cycle of a resource stored in registry.
The standard WSO2 G-reg distribution is shipped with a default ServiceLifeCycle which can be used to move service artifacts among different environments. We will modify the default ServiceLifeCycle to copy artifacts stored under /_system/dev collection to /_system/qa when promoting from development stage to testing.

Log in to G-reg management console and navigate to Extensions --> Configure --> Lifecycles. Click on Add New LifeCycle and add the following life cycle.

<aspect name="ESBLifeCycle" class="org.wso2.carbon.governance.registry.extensions.aspects.DefaultLifeCycle">
    <configuration type="literal">
        <lifecycle>
            <scxml xmlns="http://www.w3.org/2005/07/scxml"
                   version="1.0"
                   initialstate="Development">
                <state id="Development">
                    <datamodel>
                        <data name="checkItems">
                            <item name="Configurations Completed" forEvent="">
                            </item>
                            <item name="Transform Rules Done" forEvent="">
                            </item>
                            <item name="Routing Rules Completed" forEvent="">
                            </item>
                        </data>
            <data name="transitionExecution">
                            <execution forEvent="Promote" class="org.wso2.carbon.governance.registry.extensions.executors.CopyExecutor">
                           <parameter name="currentEnvironment" value="/_system/dev"/>
                           <parameter name="targetEnvironment" value="/_system/qa"/>                                
</execution>        
                        </data>
            <data name="transitionUI">
                            <ui forEvent="Promote" href="../lifecycles/pre_invoke_aspect_ajaxprocessor.jsp?currentEnvironment=/_system/dev/"/>
                        </data>
                       </datamodel>
                    <transition event="Promote" target="Testing"/>  
                   </state>  
                    <state id="Testing">
                                   
                   <transition event="Demote" target="Development"/>            
                </state>
               </scxml>
        </lifecycle>
    </configuration>
</aspect>

You can see that we have used org.wso2.carbon.governance.registry.extensions.executors.CopyExecutor class to copy one or more resources from one environment to another. Executors are custom extensions to G-reg which used to trigger a custom execution logic at the time of state transition (e.g:- Dev to QA, QA to Production). When artifacts are promoted to Testing state from Development, the CopyExecutor is triggered and the events defined under transitionExecution element are carried out. In this lifecycle, CopyExecutor copies any resource which is at currentEnvironment to targetEnvironment

Note that, CopyExecutor has not been shipped by default in WSO2 G-reg-4.1.X versions. Hence, you need to implement org.wso2.carbon.governance.registry.extensions.interfaces.Execution interface. However, in current trunk based versions (which will be released within few months), this executor is included by default and you can use it out-of-the-box.

Once the life cycle is created and saved, we can apply it to the relevant collection in registry. In this example, we need to move the artifacts stored at /_system/dev to /_system/qa. Hence, we must apply ESBLifecycle to /_system/dev collection.

Navigate to /_system/dev collection in registry browser. Click on Lifecycle in the left pane. Select Add Lifecycle and choose ESBLifeCycle from the dropdown list. Click on Add to add ESBLifeCycle to /_system/dev collection.

We do not have any artifacts in /_system/dev collection yet. Therefore, lets add some ESB artifacts through ESB management console. Log in to the management console of ESB DEV instance and add a sequence. (say it is "devsequence). This will create a new sequence in ESB and it will be stored under /_system/dev in G-reg

Now, go back to G-reg management console and navigate to /_system/dev collection. Select the check-list items such as Configurations Completed, Transform Rules Defined etc.. and click on Promote.

After promotion, click on Search --> metadata in left menu in G-reg management console and search for the resource which has just been added ("devsequence"). The resource will be copied to /_system/qa collection.

We have just completed automatic promotion of resources which have been created from ESB DEV environment to ESB QA environment. However, you will notice that the ESB sequence artifact which has just been added, is not "deployed" in ESB QA instance. Because of that, you will not see "devsequence" in sequence list of ESB QA instance. The artifact is added to the relevant registry location but in order for deployment of the resource, ESB needs to check-out the resource from /_system/qa/repository/deployment/server/synapse-configs/default/sequences location in registry to the corresponding deployment directory of file system of ESB QA server.

DeploymentSynchronizer comes into action in this situation. As I have explained in a previous blog post, DeploymentSynchronizer can be used to synchronize deployment artifacts among cluster nodes.

Lets enable registry based deployment synchronizer for ESB DEV and ESB QA instances.

- Shutdown both ESB DEV and ESB QA servers

- From WSO2 Carbon-4.0.0 release onwards, we need to enable Tribes based clustering in order for deployment synchronizer to communicate with cluster nodes. Open ESBDEV_HOME/repository/conf/axis2/axis2.xml and enable clustering and specify a unique domain name.

<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
<parameter name="domain">wso2.charitha.domain</parameter>

Repeat the same with ESBQA_HOME/repository/conf/axis2/axis2.xml as well.

- Enable deployment synchronizer configuration in ESBDEV_HOME/repository/conf/carbon.xml

<DeploymentSynchronizer>
<Enabled>true</Enabled>
<AutoCommit>true</AutoCommit>
<AutoCheckout>true</AutoCheckout>
</DeploymentSynchronizer>

- Similarly, Enable deployment synchronizer configuration in ESBQA_HOME/repository/conf/carbon.xml. ESB QA instance automatically checks out the resources from /_system/qa collection of G-reg. Therefore, we just enable AutoCheckout in that node.

<DeploymentSynchronizer>
<Enabled>true</Enabled>
<AutoCommit>false</AutoCommit>
<AutoCheckout>true</AutoCheckout>
</DeploymentSynchronizer>

We have completed enabling deployment synchronizer for both ESB development and QA servers. Now, try out adding another sequence from ESB DEV instance (say it is "devsequence2"). Once the sequence is added, access G-reg management console and browse /_system/dev collection. Promote the lifecycle again. This time, you will notice that devsequence2 is deployed in ESB QA instance upon successful promotion

In this post, I took you through the steps of moving artifacts between different environments using WSO2 ESB and WSO2 G-reg. We created artifacts in ESB using management console. However, the recommended best practice is to build SOA artifacts using a tool like WSO2 Carbon Studio and upload the CAR artifacts to ESB. We will look into CAR based SOA governance mechanism in a future post.

by Charitha at May 19, 2012 03:39 AM

May 18, 2012

Supun Kamburugamuva—Counting the code lines in a Java program

To count the code lines in a java program use the command

find . -name "*.java" -not -iwholename '*target*' -not -iwholename "*.svn*" | xargs wc -l

This will search for .java files and ignore the files in target and .svn directories.

by Supun Kamburugamuva at May 18, 2012 08:49 PM

Srinath Perera—How to scale Complex Event Processing (CEP) Systems?

What is CEP?

Complex event processing (CEP) systems query events on the fly without storing them.

For an introduction and definition of CEP, please refer to CEP Blogand Wikipedia.) .
If you need a real comprehensive coverage of CEP, read the paper "Processing flows of information: from Data stream to Complex Event Processing" [1]. (or the slide deck).

In CEP, we think in terms of event streams. Event stream is a logical sequence of events that become available over time. For example, stock event steam consists of events that notify changes to stock price. Users provide queries to the CEP engine, which implements the CEP logic, and the CEP engine matches those queries against events coming through event streams.

CEP differs from other paradigms like event processing, filtering etc., by its support for temporal queries that reason in terms of temporal concepts like "time windows" and "before and after relationships" etc. For example, a typical CEP query will say that

“If IBM stock value increased by more than 10% within an hour, please notify me".

Such a CEP query has few defining characteristics.

CEP queries generally keep running, and keep emitting events when events match the condition given in the query.
CEP query operates on the fly and stores only minimal amount of events into a storage.
CEP Engines responds to conditions generally within milliseconds range.

What is Scaling CEP?

There are many CEP Engine implementations (see CEP Players list 2012). However, mostly CEP engines run in a large box, scaling up horizontally. Vertically scaling CEP engines is still an open problem. Reminder of this post discusses what vertically scaling CEP engine means and some of the potential answers.

We use the term Scaling to describe the ability for a CEP system to hande larger or complex queries by adding more resources. Scaling CEP has several dimensions.

Handling Large number of queries
Queries that needs large working memory
Handling a complex query that might not fit within a single machine
Handling large number of events

How to Scale CEP?

Let us consider each dimension separately.

Handling Large Number of Queries

This is the easiest of the four since we can use the shared nothing architecture. Here we can run multiple CEP Engine instances (each instance runs in a single host) each running a subset of queries.

Trivial implementation will send all events (streams) to all the CEP engines
More optimized implementations can use a Publish/subscribe message broker network (like Narada Broker). Here each CEP engine should analyze the deployed queries and subscribe to required event streams in the broker network. Generally, we match each event stream to a topic in the publish/subscribe system.
Third option is to delegate the event distribution to a Stream Processing system (e.g. Apache S4 or Strom). For instance, links [4] and [5] describe such a scenario to run Esper within Strom.

Queries that need large working memory

For instance, a long running complex query that needs to maintain a large window and all events in the window would need a large working memory. Potential answer to this problem is to use a distributed cache to store the working memory. Reference [6] describes such a scenario.

Handling large number of events handling a complex query that might not fit within a single machine

We will handle the both scenarios together as both are two sides of the same coin. In both cases, we have trouble fitting a single query into a single host such that it can support the given event rate.
To handle such scenarios, we have to distribute the query across many computers.

We can do this by breaking the query into several steps in a pipeline that matches events against some conditions and republish the matching events to steps further in the pipeline. Then we can deploy different steps of the pipeline into different machines.

For example, lets consider the following query. This query matches if there are two events within 30 seconds from IBM stocks that having price greater than 70 and having prize increase more than 10%.

select a1.symbol, a1.prize, a2.prize from every 
    a1=StockStream[price > 70  symbol =’IBM’] -> 
    a2=StockStream[price > 70  symbol =’IBM’]
        [a1.price < 1.1*a2.prize][within.time=30]

As shown by the figure, we can break the query into three nodes, and each node will have to republish the matching events to the next node. (Option 1)

CEP Query as a Pipeline

However, queries often have other properties that allow further optimization. For example, although the last step of matching prize increase is stateful other two steps are stateless. Stateful operations remember information after processing an event so that earlier events affect the processing of later events while stateless operations only depends on the event being processed.

Therefore, we can add multiple instances in the place of those statless instances using a shared-nothing architecture. For example, we can break the query into five nodes as shown by the bottom part of the picture (Option 2).

Also another favorable fact is that CEP processing generally happens through filtering where amount of events reduce as we progress through the pipeline. Therefore, pushing stateless filter like operations (e.g. matching against symbol ="IBM") to the first parts of the pipeline and scaling them in shared nothing manner should allow us to scale up the system for much higher event rates. For example, lets say that the StockQuote event stream generates 100,000 events per seconds, but only 5% of them are about IBM. Therefore, only 5000 events will make it past the first filter, which we can handle much easier than 100k events.

However, it is worth noting that above method only works with some queries. For example, if we have a query that has a single stateful operation like window-based pattern, we cannot use this method.
Unfortunately, there is no framework that can do this out of the box (let me know if I am wrong). So if you want to do this, you will have to code it. If you choose to do that, using a pub/sub network or stream processing system might reduce most of the complexities.

Please shared your thoughts!
Alessandro Margara and Gianpaolo Cugola. 2011. Processing flows of information: from data stream to complex event processing. InProceedings of the 5th ACM international conference on Distributed event-based system(DEBS '11).
http://www.thetibcoblog.com/2009/08/21/cep-versus-esp-an-essay-or-maybe-a-rant/
http://www.slideshare.net/TimBassCEP/mythbusters-event-stream-processing-v-complex-event-processing-presentation
Run Esper with Storm -http://stackoverflow.com/questions/9164785/how-to-scale-out-with-esper
http://tomdzk.wordpress.com/2011/09/28/storm-esper/
Distributed Cache to scale CEP -http://magmasystems.blogspot.com/2008/02/cep-engines-and-object-caches.html

by Srinath Perera at May 18, 2012 07:53 AM

Srinath Perera—How to a measure the Performance of a Server?

I have repeated following too many times in last few years and decide to write this up. If I have missed something, please add a comment.

Understanding Server Performance

Characteristic Performance Graph's of a Server

Above graphs capture the characteristic behavior of a server. As shown by the graph, server performance is gauged by measuring latency and throughput against latency.

Latency measures the end-to-end time processing time. In a messaging environment, teams determine latency by measuring the time between sending a request and receiving the response. Latency is measured from the client machine and includes the network overhead as well.
Throughput measures the amount of messages that a server processes during a specific time interval (e.g. per second). Throughput is calculated by measuring the time taken to processes a set of messages and then using the following equation.

Throughput = number of completed requests / time to complete the requests

It is worth noting that these two values are often loosely related. However, a we cannot directly derive one measurement from the other.

As shown by the figure, a server has an initial range where throughput increases at a roughly linear rate and latency either remains constant or linear. As concurrency increases, the approximately linear relationship decays, and system performance rapidly degrades. Performance tuning attempts to modify the relationship between concurrency and throughput and/or latency, and maintain a linear relationship as long as possible.

For more details about latency and throughput, read the following online resources:

Unlike static server capacity measurements (e.g. CPU processing speed, memory size), performance is a dynamic measurement. Latency and throughput are strongly influenced by concurrency and work unit size. Larger work unit size usually negatively influence latency and throughput. Concurrency is the number of aggregate work units (e.g. message, business process, transformation, or rule) processed in parallel (e.g. per second). Higher concurrency values have a tendency to increase latency (wait time) and decrease throughput (units processed).

To visualize server performance across the range of possible workloads, we draw a graph of latency or throughput against concurrency or work unit size as shown by the above graph.

Doing the Performance Test

Your goal of running a performance test is to draw a graph like above. To do that you have to run the performance test multiple times with different concurrency and for each test measure latency and throughput.

Following are some of the common steps and a checklist.

Workload and client Setup

Each concurrent client simulates a different user. Each run in a separate thread, and run a series of operations (requests) against the server.
First step is finding a workload. If there is a well-known benchmark for the particular server you are using, use that. If not, create a benchmark by simulating the real user operations as closely as possible.
Messages generated by the test must not be identical. Otherwise, caching might come to play and provide too optimistic results. Best method is to capture and replay a real workload. If that is not possible, generate a randomized workload. Use a known data set whenever it makes sense.
We measure latency and throughput from the client. For each test run we need to measure following.
End to end time taken by each operation. Latency is the AVERAGE of all end-to-end latencies.
For each client, we collect the test-started time, test-end time, and the number of completed messages. Throughout is the SUM of throughput measured at each client.
To measure the time, if you are in Java, you can use System.nanoTime() or System. currentTimeInMillis () and with other programing languages you should use equivalent methods.
For each test run, it is best to take readings for about 10,000 messages. For example, with concurrency 10, each client should send at least 1000 messages. Even if there are many clients, each client should at least send 200 messages.
They are many tools that can do the performance test. Examples are JMeter, LoadUI, javabench, ab. Use them when applicable.

Experimental Setup

You may need to tune the server for best perforce with settings like enough Heap memory, open file limits etc.
Do not run both client and the server on the same machine (they interfere with each other and results and affected)
You need at least 1GB network to avoid the interference of the network.
Generally, you should not run more than 200 clients from the same machine. For some cases, you might need multiple machines to run the client.
You have to note down and report the environment (Memory, CPU, number of cores, operating system of each machines) with the results. It is a good practice to measure the CPU usage and memory while test is running. You can use JConsole (if it is Java) and if you are in a linux machine run “watch cat /proc/loadavg” command to track load average. CPU usage is a very unreliable matrix as it changes very fast. However, load average is a very reliable matrix.

Running the test

Make sure you restart the server between each two test runs
When you start the server, first send it few hundred requests before starting the real test to warm up the server.
Automate as much as possible. Ideally running one command should run the test, collect results, verify the results, and print summery/ graphs.
Make sure nothing else is running in the machines at the same time.
After test run has finished, check the logs and results to make sure operations were really successful.

Verifying your results

You must catch and print the errors both at the server and the client. If there are too many errors, you results might be useless. Also it is a good idea to verify the results at the client side.
Often performance tests are the first time you stress your system, and more often than not you will run into errors. Leave time to fix them.
You might want to use a profiler to detect any obvious performance bottlenecks before you actually run the tests. (Java Profilers: JProfiler, YourKit)

Analyze your results and write it up

Data cleanup (Optional) – it is a common practice to remove the outliers. General method is to either remove anything that is more than 3 X stddev different from the mean or remove 1% of furthest data from mean.
Draw the graphs. Make sure you have a title, captions for both X and Y-axis with Units, and legend if you have more than one dataset in the same graph. (You can use Excel, OpenOffice, or GNU Plot). Rule of thumb is that reader needs to be able to understand the graph without reading the text).
Optionally, draw 90% or 95% confidence intervals (Error bars)
Try to interpret what results mean

Generalize
Understand the trends and explain them
Look for any odd results and explain them
Make sure to have a conclusion

by Srinath Perera at May 18, 2012 07:48 AM

Tharindu Mathew—Useful commands for debugging OSGI issues and constraints

These are a few commands that I find very useful when faced with OSGI issues.

1. ss / ss <string> (abbr. for short status)

Lists down the state of all installed bundles or if followed by a string, does a wildcard match of that string. Useful for initial investigation of checking whether a bundle is active, installed or not present at all.

osgi> ss bam
Framework is launched.
id State Bundle 143 RESOLVED org.wso2.bam.styles_2.0.0.SNAPSHOT Master=290 150 ACTIVE org.wso2.carbon.bam.analyzer.stub_4.0.0.SNAPSHOT 151 ACTIVE org.wso2.carbon.bam.core.stub_4.0.0.SNAPSHOT 152 ACTIVE org.wso2.carbon.bam.gadgetgenwizard_4.0.0.SNAPSHOT 153 ACTIVE org.wso2.carbon.bam.gadgetgenwizard.stub_4.0.0.SNAPSHOT 154 ACTIVE org.wso2.carbon.bam.gadgetgenwizard.ui_4.0.0.SNAPSHOT 155 ACTIVE org.wso2.carbon.bam.presentation.stub_4.0.0.SNAPSHOT 156 ACTIVE org.wso2.carbon.bam2.core_4.0.0.SNAPSHOT 157 ACTIVE org.wso2.carbon.bam2.core.ui_4.0.0.SNAPSHOT 158 ACTIVE org.wso2.carbon.bam2.presentation_4.0.0.SNAPSHOT 159 ACTIVE org.wso2.carbon.bam2.receiver_4.0.0.SNAPSHOT 160 ACTIVE org.wso2.carbon.bam2.service_4.0.0.SNAPSHOT

`2. b <id> (abbr. for bundle)`


Displays details for the specified bundles. Useful for checking what your bundle exports and imports when narrowing down situations like uses constraints. Also, helps with the OSGI services used.



osgi> b 152

org.wso2.carbon.bam.gadgetgenwizard_4.0.0.SNAPSHOT [152]

Id=152, Status=ACTIVE Data Root=/Users/mackie/source-checkouts/carbon/platform/trunk/products/bam2/modules/distribution/product/target/wso2bam-2.0.0-SNAPSHOT/repository/components/configuration/org.eclipse.osgi/bundles/152/data

No registered services.

Services in use:

{org.wso2.carbon.utils.ConfigurationContextService}={service.id=123}

{org.wso2.carbon.base.api.ServerConfigurationService}={service.id=83}

Exported packages

org.wso2.carbon.bam.gadgetgenwizard.internal; version="4.0.0.SNAPSHOT"[exported]

org.wso2.carbon.bam.gadgetgenwizard.service; version="4.0.0.SNAPSHOT"[exported]

Imported packages

org.apache.commons.logging; version="1.1.1"

org.wso2.carbon.utils; version="4.0.0.SNAPSHOT"

org.wso2.carbon.user.core; version="4.0.0.SNAPSHOT"

org.wso2.carbon.registry.core.session; version="1.0.1"

org.wso2.carbon.registry.core.exceptions; version="1.0.1"

org.wso2.carbon.registry.core; version="1.0.1"

org.wso2.carbon.registry.common.services; version="1.0.1"

org.wso2.carbon.base.api; version="1.0.0"

org.osgi.service.component; version="1.1.0"

org.apache.commons.io; version="2.0.0"

org.apache.axiom.om.impl.jaxp; version="1.2.11.wso2v1"

org.apache.axiom.om.impl.builder; version="1.2.11.wso2v1"

org.apache.axiom.om; version="1.2.11.wso2v1"

javax.xml.transform.stream; version="0.0.0"

javax.xml.transform; version="0.0.0"

javax.xml.stream; version="1.0.1"

javax.xml.namespace; version="0.0.0"

net.sf.saxon; version="9.0.0.x"

No fragment bundles

Named class space

org.wso2.carbon.bam.gadgetgenwizard; bundle-version="4.0.0.SNAPSHOT"[provided]

No required bundles
3. p <package> (abbr. for packages)
Shows the bundles that export and import the specified packages. Extremely useful in debugging most OSGI issues.
osgi> p org.wso2.carbon.utils

org.wso2.carbon.utils; version="4.0.0.SNAPSHOT"

axis2_1.6.1.wso2v5 [19] imports

org.wso2.carbon.analytics.hive_4.0.0.SNAPSHOT [145] imports

org.wso2.carbon.application.deployer_4.0.0.SNAPSHOT [147] imports

org.wso2.carbon.bam.gadgetgenwizard_4.0.0.SNAPSHOT [152] imports

org.wso2.carbon.bam2.core_4.0.0.SNAPSHOT [156] imports

org.wso2.carbon.bam2.receiver_4.0.0.SNAPSHOT [159] imports

org.wso2.carbon.cassandra.dataaccess_4.0.0.SNAPSHOT [163] imports

org.wso2.carbon.cassandra.mgt_4.0.0.SNAPSHOT [164] imports

org.wso2.carbon.cluster.mgt.core_4.0.0.SNAPSHOT [169] imports

org.wso2.carbon.coordination.core_4.0.0.SNAPSHOT [172] imports

org.wso2.carbon.core_4.0.0.SNAPSHOT [173] imports

org.wso2.carbon.core.bootup.validator_4.0.0.SNAPSHOT [174] imports

org.wso2.carbon.core.services_4.0.0.SNAPSHOT [177] imports

org.wso2.carbon.dashboard_4.0.0.SNAPSHOT [178] imports

org.wso2.carbon.dashboard.common_4.0.0.SNAPSHOT [179] imports

org.wso2.carbon.dashboard.dashboardpopulator_4.0.0.SNAPSHOT [180] imports

org.wso2.carbon.dashboard.ui_4.0.0.SNAPSHOT [182] imports

org.wso2.carbon.datasource_4.0.0.SNAPSHOT [183] imports

org.wso2.carbon.event.client_4.0.0.SNAPSHOT [187] imports

org.wso2.carbon.event.common_4.0.0.SNAPSHOT [189] imports

org.wso2.carbon.event.core_4.0.0.SNAPSHOT [190] imports

org.wso2.carbon.event.ws_4.0.0.SNAPSHOT [191] imports

org.wso2.carbon.jaggery.app.mgt_1.0.0.SNAPSHOT [222] imports

org.wso2.carbon.jaggery.app.mgt.ui_1.0.0.SNAPSHOT [224] imports

org.wso2.carbon.jaggery.deployer_1.0.0.SNAPSHOT [226] imports

org.wso2.carbon.logging.service_4.0.0.SNAPSHOT [232] imports

org.wso2.carbon.ndatasource.core_4.0.0.SNAPSHOT [236] imports

org.wso2.carbon.ntask.core_4.0.0.SNAPSHOT [239] imports

org.wso2.carbon.registry.common_4.0.0.SNAPSHOT [246] imports

org.wso2.carbon.registry.core_4.0.0.SNAPSHOT [248] imports

org.wso2.carbon.registry.resource.ui_4.0.0.SNAPSHOT [254] imports

org.wso2.carbon.registry.server_4.0.0.SNAPSHOT [258] imports

org.wso2.carbon.registry.servlet_4.0.0.SNAPSHOT [259] imports

org.wso2.carbon.reporting.template.core_4.0.0.SNAPSHOT [263] imports

org.wso2.carbon.security.mgt_4.0.0.SNAPSHOT [273] imports

org.wso2.carbon.security.mgt.ui_4.0.0.SNAPSHOT [275] imports

org.wso2.carbon.server.admin_4.0.0.SNAPSHOT [276] imports

org.wso2.carbon.server.admin.ui_4.0.0.SNAPSHOT [279] imports

org.wso2.carbon.service.mgt_4.0.0.SNAPSHOT [280] imports

org.wso2.carbon.transport.http_4.0.0.SNAPSHOT [285] imports

org.wso2.carbon.transport.https_4.0.0.SNAPSHOT [286] imports

org.wso2.carbon.transport.mgt_4.0.0.SNAPSHOT [287] imports

org.wso2.carbon.ui_4.0.0.SNAPSHOT [290] imports

org.wso2.carbon.user.core_4.0.0.SNAPSHOT [295] imports

org.wso2.carbon.user.mgt.ui_4.0.0.SNAPSHOT [299] imports

org.wso2.carbon.webapp.mgt_4.0.0.SNAPSHOT [301] imports

org.wso2.carbon.wsdl2form_4.0.0.SNAPSHOT [302] imports
4. diag <bid> (abbr. for diagnose)
Shows any unsatisfied constraints of the bundle.



osgi> diag 159

reference:file:plugins/org.wso2.carbon.bam2.receiver_4.0.0.SNAPSHOT.jar [159]

No unresolved constraints.
 
5. ls (abbr. for list services)
Lists down the state of all OSGI services. In this list the most important would be identifying the unsatisfied components as in component 20 below.
 osgi> ls

All Components:

ID State Component Name Located in bundle

1 Registered org.eclipse.equinox.frameworkadmin.equinox org.eclipse.equinox.frameworkadmin.equinox(bid=108)

2 Active org.eclipse.equinox.p2.artifact.repository org.eclipse.equinox.p2.artifact.repository(bid=114)

3 Active org.eclipse.equinox.p2.core.eventbus org.eclipse.equinox.p2.core(bid=116)

4 Active org.eclipse.equinox.p2.di.agentProvider org.eclipse.equinox.p2.core(bid=116)

5 Registered org.eclipse.equinox.p2.director org.eclipse.equinox.p2.director(bid=117)

6 Active org.eclipse.equinox.p2.planner org.eclipse.equinox.p2.director(bid=117)

7 Active org.eclipse.equinox.p2.engine.registry org.eclipse.equinox.p2.engine(bid=120)

8 Active org.eclipse.equinox.p2.engine org.eclipse.equinox.p2.engine(bid=120)

9 Active org.eclipse.equinox.p2.garbagecollector org.eclipse.equinox.p2.garbagecollector(bid=122)

10 Active org.eclipse.equinox.p2.metadata.repository org.eclipse.equinox.p2.metadata.repository(bid=125)

11 Registered org.eclipse.equinox.p2.repository org.eclipse.equinox.p2.repository(bid=128)

12 Registered org.eclipse.equinox.p2.transport.ecf org.eclipse.equinox.p2.transport.ecf(bid=132)

13 Registered org.eclipse.equinox.p2.updatechecker org.eclipse.equinox.p2.updatechecker(bid=133)

14 Registered org.eclipse.equinox.simpleconfigurator.manipulator org.eclipse.equinox.simpleconfigurator.manipulator(bid=138)

15 Active bam.hive.component org.wso2.carbon.analytics.hive(bid=145)

16 Active application.deployer.dscomponent org.wso2.carbon.application.deployer(bid=147)

17 Active gadgetgenwizard.component org.wso2.carbon.bam.gadgetgenwizard(bid=152)

18 Active bam.utils.component org.wso2.carbon.bam2.core(bid=156)

19 Active bam.presentation.component org.wso2.carbon.bam2.presentation(bid=158)

20 Unsatisfied bam.receiver.component org.wso2.carbon.bam2.receiver(bid=159)

21 Active org.wso2.carbon.cassandra.dataaccess.component org.wso2.carbon.cassandra.dataaccess(bid=163)

22 Active org.wso2.carbon.cassandra.mgt.component org.wso2.carbon.cassandra.mgt(bid=164)
6. comp <component id> or ls -c <bundleid>
Lists component specific information regarding OSGI declarative services. Useful for debugging issues with declarative services.



osgi> ls -c 152

Components in bundle org.wso2.carbon.bam.gadgetgenwizard:

ID Component details

17 Component[

name = gadgetgenwizard.component

factory = null

autoenable = true

immediate = true

implementation = org.wso2.carbon.bam.gadgetgenwizard.internal.GadgetGenWizardServiceComponent

state = Unsatisfied

properties = {service.pid=gadgetgenwizard.component}

serviceFactory = false

serviceInterface = null

references = {

Reference[name = config.context.service, interface = org.wso2.carbon.utils.ConfigurationContextService, policy = dynamic, cardinality = 1..1, target = null, bind = setConfigurationContextService, unbind = unsetConfigurationContextService]

Reference[name = server.configuration, interface = org.wso2.carbon.base.api.ServerConfigurationService, policy = dynamic, cardinality = 1..1, target = null, bind = setServerConfiguration, unbind = unsetServerConfiguration]

}

located in bundle = org.wso2.carbon.bam.gadgetgenwizard_4.0.0.SNAPSHOT [152]

]

Dynamic information :

The component is satisfied

All component references are satisfied

Component configurations :

Configuration properties:

service.pid = gadgetgenwizard.component

component.name = gadgetgenwizard.component

component.id = 16

Instances:

org.eclipse.equinox.internal.ds.impl.ComponentInstanceImpl@3fabb84d

Bound References:

String[org.wso2.carbon.utils.ConfigurationContextService]

-> org.wso2.carbon.utils.ConfigurationContextService@22d0e7e3

String[org.wso2.carbon.base.api.ServerConfigurationService]

-> org.wso2.carbon.base.ServerConfiguration@4127f9f0


by Mackie Mathew at May 18, 2012 06:57 AM




Evanthika Amarasiri—How to resolve 'ORA-01882: timezone region not found' when starting WSO2 product servers with Oracle
Have you ever come across the below issue when starting any of the WSO2
Carbon servers,pointing to an Oracle database WITH ojdbc6.jar?Caused by: org.wso2.carbon.user.core.UserStoreException:
Database Error - Cannot create PoolableConnectionFactory
(ORA-00604: error occurred at recursive SQL level 1ORA-01882: timezone region
not found) at org.wso2.carbon.user.core.claim.dao.ClaimDAO.getDialectCount
(ClaimDAO.java:168) at org.wso2.carbon.user.core.common.DefaultRealm.
populateProfileAndClaimMaps(DefaultRealm.java:278) at
org.wso2.carbon.user.core.common.DefaultRealm.init(DefaultRealm.java:109)
at org.wso2.carbon.user.core.common.DefaultRealmService.initializeRealm
(DefaultRealmService.java:240) ... 17 moreCaused by:
org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory
(ORA-00604: error occurred at recursive SQL level 1ORA-01882:
timezone region  not found) at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory
(BasicDataSource.java:1549) at org.apache.commons.dbcp.BasicDataSource.createDataSource
(BasicDataSource.java:1388) at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
at org.wso2.carbon.user.core.claim.dao.ClaimDAO.getDialectCount(ClaimDAO.java:158)
... 20 moreWell, if you have, all you have to do is, set the below JAVA property
in the terminal that you are running the server on, before starting the
server (+05:30 if you are in Sri Lanka. May vary from the location that
you are located at).export JAVA_OPTS="-Duser.timezone=='+05:30'"The above setting will only take effect to the current terminal. Therefore, if you need to set this property permanently, all you have to do is, add it to the wso2server.sh as a new JAVA_OPT property (See below).   

by Evan at May 18, 2012 02:20 AM

May 17, 2012


Nuwan Bandara—Bandos’ Arcade, gets a new look and feel



I need to thank few. Jade Dominguez for developing the  “the-program” theme, I grabbed the idea from it. A huge thank to Teague, the awesome designer who gave me some tips and made me a cartoon of myself :P
by Nuwan Bandara at May 17, 2012 04:29 PM

May 16, 2012


Tharindu Mathew—Solving the XML Parsing error in Firebug
XML Parsing Error: no element found Location: moz-nullprincipal:{e04285c3-86a1-f146-b683-55bb667191ea} Line Number 1, Column 1:
^
If you are getting the above error in firebug, you are most probably making a cross browser request that is not allowed and not having invalid XML in your response. Cross browser requests are not allowed, unless you use jsonp or script as the data type.
To fix this problem, make sure the url doesn’t start with a http/ https:
 $.ajax({ url : "http://localhost/somewhere" })
This is not allowed, unless you follow the rules to make a cross browser request. You probably want to do something like:
 $.ajax({ url : "somewhere" })

         
by Mackie Mathew at May 16, 2012 06:11 AM


Afkham Azeez—Scatter-Gather EIP with WSO2 ESB








Scatter-Gather or Broadcast Aggregate is a popular Enterprise Integration Pattern (EIP). The example given in the EIP catalog talks about a scenario where you send a quote request to a 3 vendors, and get quotes for certain items, and send back the response from the vendor who provides the best price.



We will show how this can be simulated using the WSO2 ESB, , which is the fastest, proven & best performing & most flexible open source ESB. For the sake of convenience, we will take the SimpleStockQuoteService as the service provided by the vendors, and return the best quote to the client. 




We will use 3 constructs in the ESB to implement this scenario.


Clone mediator
Aggregate mediator 
Enrich mediator

We will write a simple proxy service called ScatterGatherProxy. The assumption here is that all 3 vendors implement the same service contract. If the service contracts are different, we will first have to do the appropriate transformations before sending the requests to the vendor services, and then transform back the responses. The XSLT mediator can be used to do that. The configuration of this proxy service is shown below.



<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse" name="ScatterGatherProxy" transports="https http" startOnLoad="true" trace="disable">
    <description/>
    <target>
        <inSequence>
            <clone>
                <target>
                    <endpoint name="vendorA">
                        <address uri="http://localhost:9000/services/SimpleStockQuoteService/"/>
                    </endpoint>
                </target>
                <target>
                    <endpoint name="vendorB">
                        <address uri="http://localhost:9001/services/SimpleStockQuoteService/"/>
                    </endpoint>
                </target>
                <target>
                    <endpoint name="vendorC">
                        <address uri="http://localhost:9002/services/SimpleStockQuoteService/"/>
                    </endpoint>
                </target>
            </clone>
        </inSequence>
        <outSequence>
            <log level="full"/>
            <aggregate>
                <completeCondition>
                    <messageCount min="3"/>
                </completeCondition>
                <onComplete xmlns:m1="http://services.samples/xsd" xmlns:m0="http://services.samples" expression="//m0:return">
                    <enrich>
                        <source xmlns:m1="http://services.samples/xsd" clone="true" xpath="//m0:return[not(preceding-sibling::m0:return/m1:last &lt;= m1:last) and not(following-sibling::m0:return/m1:last &lt; m1:last)]"/>
                        <target type="body"/>
                    </enrich>
                    <send/>
                </onComplete>
            </aggregate>
        </outSequence>
    </target>
</proxy>



To be able to run this sample, you will need to deploy the SimpleStockQuoteService on the Sample AxisServer which ships with the WSO2 ESB, and start 3 instances of it, with HTTP ports 9000, 9001 & 9002 respectively.



Next deploy the above proxy service on WSO2 ESB.



Now let's use soapUI to send the following request to the ScatterGatherProxy service.



<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ser="http://services.samples">
   <soapenv:Header/>
   <soapenv:Body>
      <ser:getSimpleQuote>
         <ser:symbol>IBM</ser:symbol>
      </ser:getSimpleQuote>
   </soapenv:Body>
</soapenv:Envelope>





Since we have enabled the log mediator inside the outSequence, you will see 3 responses from the 3 vendors. The logs you will see will be similar to the following







In soapUI, you will get the response from the vendor who provided the best quote.







If you carefully compare the logged response messages & the response received by the client, you will notice that the ScatterGatherProxy service returns the best quote to the client (soapUI)







Underneath the Hood
So what is going on in this proxy service?



Let us first look at the inSequence of the ScatterGatherProxy service. In the inSequence, we are simply cloning, or making 3 copies of the request that was sent by the client using the Clone mediator, and forwarding those requests to the 3 vendor services (SimpleStockeQuoteService).



The responses to those 3 requests will be received at the outSequence. Here we first log all the received responses, and then use the Aggregate mediator  to aggregate the responses. The onComplete sequence of the Aggregate mediator   will get called once all 3 responses are received or the specified completion condition is met. We aggregate the responses based on the value of the "return" element in the response. Next we use the Enrich mediator to extract the response which contains the best quote. We use the following XPath 1.0 expression to do this;



//m0:return[not(preceding-sibling::m0:return/m1:last <= m1:last) and not(following-sibling::m0:return/m1:last < m1:last)]



This is a complex way of saying, "pick the response which has the lowest 'last' value". Unfortunately, XPath 1.0, which is the default supported by WSO2 ESB, does not support the XPath 2.0 min function which can reduce the complexity of the above statement by a great deal. Once we find the proper response, we enrich the SOAP body with that response, and then send that response back to the client (soapUI)

by Afkham Azeez at May 16, 2012 05:12 AM


Chris Haddad—Re-invent Software Delivery and Offer Your Business as a Service
As business leaders focus on growth during 2012, they are identifying business expansion and transformation opportunities.  The resulting IT mandate to rapidly evolve mobile and social interactions is forcing CIOs to re-invent their software delivery.  By following a straightforward four-step plan, CIOs can improve productivity, enhance agility, deliver timely solutions, and help fulfill strategic business growth goals.

Strategic business growth goals often include delivering new lines of business, selling through new distribution channels, expanding into new geographical markets, simplifying business workflow, delivering services directly to customers, and interacting through mobile channels.  A limited capital environment and short investment timeframe requires building a flexible business service model, which relies on customizing existing products and operational models.
The execution plan should highlight capability configuration and rapid, efficient business service delivery.  Business leaders often specify ‘branch in a box’, ‘localization and centralization’, and ‘on-demand self-service’ terms to indicate doing more with less, fulfilling local compliance regulations while preserving core business processes, and quickly delivering a comprehensive solution into remote areas.
Today’s business evolution pace reduces the optimal time to market and decreases the available IT execution window.  In many organizations, re-inventing software delivery is a pre-requisite before effectively executing business strategies.  CIOs are tasked with determining how IT technology, processes, and organizational models must change.  When determining whether your IT organization should re-invent software delivery to enhance execution, consider your existing architecture flexibility, team productivity, and historical time to market.
For example, a well-known business supply company asked the IT team to deliver a mobile application within 90 days.  The business required a complete, mobile shopping and checkout experience mirroring the company’s online ecommerce website.  Unfortunately, the existing application architecture lacked critical, required functionality inside the website.  The code base exposed only 50 percent of the required ecommerce business processes.  One year later, the company finally released the mobile application, and the business viewed the CIO’s team as a market growth inhibitor.
A CIO for a property and casualty insurance company authorized a service-oriented architecture (SOA) initiative in September 2003.  Seven years later, the CIO was being questioned about a $3 million yearly budget line item.   The line item funded the IT-led SOA initiative.  The CIO’s directors could not quantify the business value derived from the SOA initiative and stated that the initiative would take three more years (and an additional $9 million) to complete.  The business continually questioned the IT team’s service development productivity and fiscal responsibility.
A third CIO leading a financial services company owned a monolithic, legacy Web application architecture.  Although the architecture was only four years old, the architecture impeded the organization’s ability to deliver new lines of business and match the expected business volume.  Executive management was hesitant to migrate the application to another platform due to the business disruption encountered during the last platform rollout.  The IT team was unable to quantify the risk and value expected by a new platform initiative, and executive management perceived the CIO as pursuing new technology for technology sake.
Re-invent software delivery
All CIOs must continually lead their teams to improve how projects are designed, developed, deployed and measured.  Transformation charters commonly specify improvement through agile development methodologies, outsourcing, deploying pre-packaged applications, implementing SOA, or adopting platform-as-a-service (PaaS).   When scaling IT initiatives to meet business goals, a strategic, CIO-led technology plan is often required to overcome organizational inertia and re-invent software delivery.
Consider the following four-step strategic technology plan to re-invent the software delivery process:

Identify core business capabilities and outsource commodity.
Iteratively build extensible, configurable business services, APIs, and applications.
Increase business partnerships with API management that provides on-demand, self-service and analytics.
Host services, APIs and applications on a multi-tenant platform and facilitate per-tenant configuration

Step 1. Identify core business capabilities and outsource commodity
When teams gain consensus on their core business capabilities and define non-core, commodity processes, they have established a foundation to focus development effort and increase IT impact.  Non-core, commodity processes should be outsourced and supported by packaged applications.  Reducing bespoke, custom development frees resources and directs the team towards generating meaningful competitive advantage.  Business capability modeling is a technical discipline closely aligned with business process improvement.  CIOs should allocate a few smart architects to work with existing business process improvement experts (e.g. Six Sigma Black Belts, Kanban experts) and track core business capabilities.
A registry can be used to track the intersection between business capabilities and application assets.  Ideally, the registry would be linked with the operational configuration management database (CMDB) to provide a complete view of how business capabilities are instantiated across the IT environment.
Step 2. Iteratively build extensible, configurable business services, APIs, and applications
Pre-built functionality reduces time, effort and resources required to field new solutions.  When building services and applications, the architectural principles of extensibility and configurability will create opportunities to adapt solutions for new geographies, product lines, and markets.   Architects must be adept at identifying the commonality from the variability, and teams must iteratively build solutions to evolve and endure, rather than build for single use or to throw away.  Effective CIOs establish architecture review and governance boards to consistently evaluate development, align extension points with the business strategy, and iteratively enhance the portfolio.
When the organization identifies strategic growth opportunities in developing economies and new markets, application architecture must shift from silos and toward services.  Silos decrease business efficiency by fragmenting the user experience, increasing data management challenges, and fostering process isolation. Agile methodologies highlight inclusive engagement and user feedback.  With the correct focus, development initiatives can unify workflow, foster collaboration, facilitate consistency, and promote solution cohesion.  A complete, composable, cohesive, and interoperable middleware platform will encourage holistic service development. 

Step 3. Increase business partnerships with API management’s on-demand self-service and analytics
Many CIOs experience rapid portfolio proliferation and sprawl, but not enhanced portfolio efficiency or business agility.  Achieving business agility requires the growth of development partnerships and interactions, which should span both internal and external teams.
Traditional SOA and integration platforms enable rapid development, but they provide little business partnership support.  Teams commonly operate independently and autonomously.   Hundreds of people write new APIs and services; few people know:

who is consuming APIs and services,
who is writing re-usable APIs and services, or
how APIs and services are being used.

Teams must improve cross-team (or cross-partner) communication, coordination and collaboration.  CIOs should encourage their teams to extend the governance registry and offer managed APIs through an API Store.  A managed API is:

Actively advertised and subscribe-able
Available with an associated, published service-level agreement (SLA)
Secured, authenticated, authorized and protected
Monitored and monetized with analytics

An API Store is a venue to find, explore, subscribe and evaluate available resources.  The API Store enables partners to quickly find relevant APIs.  Once a candidate list is identified, the API Store provides a structured environment for exploring the APIs and understanding solution fit.   During the exploration phase, collaboration between the potential API consumer and provider is essential.  After finding and exploring an API, a project may stall when the team attempts to gain access.  An API store provides on-demand self-service subscription and collaboration channels, rapidly reducing the time and effort required to integrate and evaluate available API resources. Figure 1 illustrates API consumer lifecycle activities.

Figure 1: API Consumer Lifecycle Activities
When selecting APIs, trust is an important consideration.  Without trust, potential business partners will choose other alternatives or build their own solution.  CIOs must establish an environment where their team is the trusted provider of choice.  When teams follow best practices, partners recognize competency and reduce their adoption trepidation.
Teams increase competency when they establish a separation between API provider and API manager responsibilities.  An API provider is responsible for building, publishing, scaling and versioning the API.  The API manager is focused on promoting and encouraging potential consumers to adopt the API.   The manager analyzes usage patterns and determines how to best monetize the asset.  A monetization strategy solve a perennial IT questions:

“Once I offer an API, what should be the show-back, charge-back mechanism?”
“How do I actually perform investment re-capture activities?”

Step 4. Host services, APIs and applications on a multi-tenant platform and facilitate per-tenant configuration
Offering a business capability as a one-size-fits-one API is a typical IT solution trap.  One-size-fits-one solutions do not exhibit the adaptability or agility required to fulfill new business opportunities.   CIOs are intrigued by the cloud’s promise to create a one-size-fits-ALL solution.  Cloud characteristics and vertical PaaS accelerate the IT team’s ability to deliver solutions that support business growth objectives.
Cloud characteristics advance a company’s ability to offer business capabilities as on-demand services.  Cloud characteristics describe IT’s ability to deliver on-demand self-service, rapid elasticity, resource pooling, and measured service.  Figure 2 associates cloud characteristics with architectural goals.

Figure 2: NIST Cloud Characteristics and associated architectural goals
On-demand self-service and resource pooling will flexibly assign workloads and decrease provisioning periods.  If teams excessively customize an environment, they can increase time to market, lower resource pooling, and create a complex, one-size-fits-one environment, which is difficult to manage and maintain.  A CIO should encourage a governance process that minimizes exceptions.  Consumers in a one-size-fits-all environment will predominantly subscribe to standard service offerings.
Business users don’t really care how many server instances are running in the cloud.  Business users care about business entities, business activity performance, and associated cost.  A CIO who decouples metering and billing from IT assets and shifts the reporting model to focus on business activity cost will positively transform investment conversations.
When you pre-build vertical industry APIs and vertical industry components, you can decrease time to market, bring new partners online, and create new revenue sharing opportunities.   With a vertical PaaS, your organization offers vertical business capabilities using a multi-tenant, extensible cloud environment.  The environment decreases time to market for partners, creates new ecosystem scenarios, and enables revenue sharing opportunities.
The vertical PaaS environment also provides an opportunity for your partners to deeply embed your business capabilities within their applications (similar to Force.com, eBay sellers, or Amazon Store environments).  By hosting all business partners (e.g. suppliers, customers and employees) within a multi-tenant environment, the environment can easily aggregate and share business information. Figure 3 illustrates how a complete middleware platform, API management, and vertical PaaS deliver an ecosystem platform.
 Figure 3: Vertical Platform as a Service Environment
Summary
Today’s business evolution pace reduces the optimal time to market and decreases the available IT execution window.  The resulting IT mandate to rapidly offer core business capabilities as configurable services is forcing CIOs to re-invent their software delivery.  A strategic plan can keep your team on track.  Guide your team towards identifying core business capabilities, building extensible and configurable business services, increasing business partnerships, and building a business ecosystem.  As your business ecosystem emerges, your organization will encounter new revenue sharing opportunities, and an opportunity for your partners to leverage your core capabilities in ways that you haven’t yet envisioned.
 

by haddadcblg at May 16, 2012 12:44 AM


May 15, 2012


Afkham Azeez—Dead Letter Channel EIP with WSO ESB





Figure 1: Dead Letter Channel in action




Dead Letter Channel (DLC) is an Enterprise Integration Pattern (EIP) that outlines how messaging systems could deal with messages that cannot be delivered to the recipient. Due to system failures, network failures or failures at the recipient, messages may not get delivered to the target recipient. In such a scenario, the messaging system can deliver the message to a DLC. Other mechanisms in the DLC implementation should take care of trying to deliver the dead message to the recipient. This could be done by periodically retrying to send the message to the recipient over a defined time period. Persistence of the dead message could also be another option provided by the DLC implementation, so that even if the messaging system fails, once it is rebooted, the dead messages can be delivered to the receivers.




WSO2 ESB, which is the fastest, proven & best performing & most flexible open source ESB available; implements the DLC pattern using a combination of 3 constructs;





store mediator
message stores
message processors



Figure 2: DLC architecture in WSO2 ESB


As shown in the figure above, the store mediator takes care of storing the dead message in the specified message store. WSO2 ESB contains two message store types, in memory & JMS. In addition, users can provide their own custom message store implementations. The message processor will retrieve the stored messages from the message store associated with it & try to resend those messages to the target receiver. The message store & message processor combination act as the dead letter channel.




Charith has explained in detail how you could configure message stores & message processors  in WSO2 ESB in his article; Implementing Store and Forward Messaging Patterns with WSO2 ESB Part 2

by Afkham Azeez at May 15, 2012 12:47 PM


Buddhika Chamith—Learn by Errors : Java + OSGi
Recently I worked on getting Apache Hive work inside an OSGi environment. While

not proving to be a proverbial piece of cake (software right?.. Why am I not

surprised?  ), it led me through an assortment of Java and OSGi errors. Here I

am listing some of them that bit me bit hard (no pun intended) so that I

thought of making a blog out them just for my own satisfaction.
java.lang.VerifyError
I got this nastiness during initialization of one of OSGi service components.

The culprit was not immediately identifiable since the offending bundle was in

ACTIVE state. On the surface everything looked fine except for the fact the

Hive server which was supposed to start during the initialization of the

service component present in the bundle was not up and running. A quick ‘ls’ in

the OSGi console revealed the service component is in ‘unsatisfied’ state.

Finally a ‘comp’ revealed the root cause, the VerifyError.
The VerifyError can occur if the runtime dependency of a class is different to that

of the dependency that was used at compilation time. For example if the method

signatures have changed between the dependencies then this error would result.

This is nicely explained at [1] in the accepted answer. As it turned out

slightly different versions of a package had been exposed in two bundles causing

the Hive bundle to pick up a different version over the version that was in the

compilation environment. Proper OSGi versioning turned out to be the solution.
java.lang.IncompatibleClassChangeError
This error also cropped up under a similar circumstance where two packages were

present in the system. As [2] clearly explains, the reason for this in my case

was an interface being changed to an abstract class between the conflicting

package versions. Again the versioning helped to save the day.
java.lang.LinkageError : loader constraint violation in xxxx – blah …
Now this seems to be a famous error specially in OSGi enviornments. Main root

cause seems to be two classes loaded by different ClassLoaders coming in to

contact in a certain way. For example say Class A object accept a Class B object

as a method parameter. Class B is loaded by ClassLoader-A which also loads Class

A. But at the method invocation time how ever an object of Class B which has

been loaded by ClassLoader-B is passed as an argument to an object of Class A

which has been loaded by ClassLoader-A. Now the result would be a big fat

LinkageError with a very verbose error message.
The graph based class loadingstructure in OSGi makes it specially conducive to these kind of errors. In my case the culprit was a package which had been duplicated in two different

bundles and a particular class in that package loaded by the separate

ClassLoaders of each of the bundles coming in to contact via a third bundle

present in the system during a method call. So this was a case of not following

“import what you export” best practice [3] in OSGi. Doing so would help to

reduce the exposure of duplicated packages across bundles and help to maintain a

consistent class space for a given package. And so this turned out to be the

resolution for that in this case.
Package uses conflict: Import-Package: yyy; version=”x.x.x”
I had my fair share of this inconvenience thrown at my face every so often

during the exercise. There are two excellent posts [4],[5] exactly on this issue

at SpringSource which helped a lot. However let me summarize my learning on this

issue. Simply if a bundle is being exposed to two versions of the same package

through a direct import and via a uses constraint this error would come up. The

diagram best illustrates this situation.

The bundle A imports org.foo version 1.0.0 directly. However it also imports

bundle org.bar from bundle B. However as it turns out package org.bar also uses

org.foo package albeit it’s a different version (2.0.0) than that of the version

imported by bundle A. Now bundle A is directly wired to version 1.0.0 of org.foo

and also being exposed to the version 2.0.0 of org.foo due to the

import of org.bar which is using version 2.0.0 of org.foo. Now since a bundle

cannot be wired to different versions of the same package, a uses conflict would

come up with offending import org.bar as the root cause. (e.g: Package uses

conflict: Import-Package: org.bar; version=”0.0.0″). The solution would be to

change package import versions of org.bar in either bundle A or bundle B so that

both would be pointing to the same package version. Another excellent blog by

Neil Bartlett on this can be found at [6].
java.lang.UnsatisfiedLinkError
One of my friends at work came across this while trying to incorporate another

third party library in to our OSGi enviornment. JavaDocs goes on to say that

this gets “Thrown if the Java Virtual Machine cannot find an appropriate

native-language definition of a method declared native”. The offending library

was a linux .so (dynamically linked library) file which was not visible to

bundle ClassLoader at runtime. We were able to get it working by directly

including the library resource to the bundle ClassLoader. An earlier attempt on

setting this resource on TCCL (Thread Context ClassLoader) failed and this let

us to the realization that the TCCL is typically not the bundle class loader. A

good reading on TCCL under Equinox OSGi enviornment can be found at [7].
 
[1] http://stackoverflow.com/questions/100107/reasons-of-getting-a-java-lang-verifyerror

[2] http://stackoverflow.com/questions/1980452/what-causes-java-lang-incompatibleclasschangeerror

[3] http://blog.osgi.org/2007/04/importance-of-exporting-nd-importing.html

[4] http://blog.springsource.org/2008/10/20/understanding-the-osgi-uses-directive/

[5] http://blog.springsource.org/2008/11/22/diagnosing-osgi-uses-conflicts/
[6] http://njbartlett.name/2011/02/09/uses-constraints.html

[7] http://wiki.eclipse.org/ContextClassLoader_Enhancements

         
by Buddhika Chamith at May 15, 2012 10:06 AM

May 14, 2012


Kathiravelu Pradeeban—Moving from a 'Platform' to the 'Platform-as-a-Service' ~ What is it all about?


Carbon Middleware Platform for each department.


Cloud Middleware Platform 

In a traditional system, we see each department is equipped with its own middleware platform, having multiple servers for each of the departments. With the Cloud Middleware Platforms (CMP), a single product which is installed  and managed centrally, can replace the entire set of platforms  installed separately, say for each departments. Here each department is considered a "tenant". 



Cloud Enablement

Are all the application platforms  are cloud enabled? No, obviously not. The cloud enablement comes with  the fruits of multi-tenancy, and the native support to be a cloud  platform. An  ideal example would be, WSO2 Carbon Platform, where the same platform,  cloud enabled, becomes WSO2 Stratos Cloud Middleware Platform. A cloud  middleware platform can usually be hosted over the cloud as a Platform  as a Service. The publicly hosted Stratos Cloud Middleware Platform is StratosLive (stratoslive.wso2.com), which allows anyone to register them an account (tenant). It should be noted that not all the PaaS are provided as a  fully functional cloud middleware platforms. 



Tenant Isolation


Stratos Cloud Middleware Platform


Tenants, while sharing the same resource, will not be aware of the existence of the other tenants. Here, multi-tenancy enables centrally installed/deployed and managed resources, providing an independent middleware platform virtually for each department, registered as a tenant, in the organization's cloud middleware platform. Each tenant can have multiple users.



Centrally managed! 

Though it is just a single server that is running, each tenant is properly isolated, such that they will feel that they have a separate instance of the platform. The isolation is achieved at data and logic level, whilst providing the relevant security for the tenants. The security challenges are addressed by the security measures, where the tenants are not allowed to execute the privileged operations such as writing to the file system or opening up a port. A few of the management operations are limited to the administrator of the system - termed as Super tenant in WSO2 Stratos, which becomes the central point to manage all the tenants.



 



Bob meets Alice - Cloud Middleware Platform

SaaS Development over CMP


A cloud middleware platform, as the application platforms, can be hosted in local data centers, or even on personal computers that have the required amount of disk space and  memory. This can be used to incrementally test and develop the SaaS applications locally, than developing and hosting them directly on the cloud. 



Platform as a Service 

A cloud middleware platform, when is hosted on a private, public, or a hybrid cloud infrastructure, becomes a Platform as a Service. Cloud Middleware Platform is also referred to as a cloud enabled application platform (CEAP), by Gartner, indicating that the application platform is cloud-enabled. 



WSO2 StratosLive

WSO2 StratosLive, the open java PaaS, is the publicly hosted cloud deployment of Stratos, from WSO2. Migration of your data and the applications between the PaaS, and the Cloud Middleware Platform hosted on your private data center is generally an easy job. WSO2 provides it from the bottom up - from an enterprise middleware platform named as Carbon, to the cloud middleware platform named as Stratos, and finally to the Platform as a Service - StratosLive and the other public, private, or hybrid clouds with Stratos as the Cloud Middleware Platform.



PaaS and ROI



Cloud Middleware Platform and Paa

Why should a Software as a Service developer/provider go for an PaaS provider, instead of hosting their applications directly on IaaS? The PaaS layer that stays between the applications and the infrastructure should provide value to the enterprise, to answer the above question. Moreover, a PaaS can also be hosted directly on the native hardware as opposed to hosting them on top of IaaS, as the commonly mentioned bottom up cloud architecture of IaaS -> PaaS -> SaaS. This prevents the application developer worrying about the underlying infrastructure or hardware when developing his applications. 



PaaS for SMEs

For a start-ups or small and medium enterprises, a suitable PaaS provides faster time-to-market, providing higher return on investment (ROI). When you are hosting an application over the Platform as a Service,  the platform should be able to handle the high-availability, fail over, auto-scaling, logging, throttling, and billing features. This eliminates the need for the application developers to code for these common requirements which to be available in the platform level. WSO2 StratosLive, as a complete middleware platform as a service, provides an entire architecture as a service. This made possible since Stratos/StratosLive is extending the WSO2 Carbon SOA enterprise middleware platform, sharing the same code base.



No Code for a platform or infrastructure!



PaaS handles them all!

An open platform as a service is committed to fight against the vendor lock-in, by adhering to the open standards. Open source technologies help a lot in being committed to being open. Being open means, the application developers should not be writing an application solely focusing a platform or the API provided by the cloud infrastructure or platform providers. WSO2 is open by design.






This blog post has been published on WSO2 Library.



Resources:

Concerns of the public cloud and how PaaS helps mitigating them...


Summer School 2011 - Platform-as-a-Service: The WSO2 Way

- Presentation Slides

- Recording

Building SaaS for SMEs on WSO2 PaaS - Wednesday, 9th November 2011

WSO2 StratosLive - An Enterprise Ready Java PaaS

Auto-Scaling with WSO2 Load Balancer.

StratosLive Frequently Asked Questions

84 Open Source Companies Ranked By Momentum Index

by Kathiravelu Pradeeban at May 14, 2012 09:10 PM



Saliya Ekanayake—Recursion is Natural
 
  
 “Yay! I can see my bu** !! … wait, it’s not my bu**, it’s myself !”
 If you can come to the last realization that you are seeing yourself then you already understand recursion is natural. If not, the following few examples may help.
 Example 1: Is Even?
 Zero is even. Any integer N > 0 is even if N-1 is not even.
 Example 2: Factorial
 Factorial of zero is 1. Factorial of any integer N > 0 is N times factorial of N-1.
 Example 2: Length of a List
 An empty list has a length zero. Any other list has one head element and a sub list called the tail. So length is 1 more than the length of the tail.
 Example 3: Map f() to List S
 If S is empty then nothing to do just return an empty list. If not map f() to the tail and get a mapped list. Then add f(head) to the front of that list.
  
 P.S. Many thanks to Dan Friedman (https://www.cs.indiana.edu/~dfried/) for his great class of B521 (cs.indiana.edu/classes/b521) at IU, 2009.
 ----------
 About the image; it’s an InkScape sketching of the image I found at http://contortionistsunite.ning.com/profiles/blogs/day-1-3
  
by Saliya Ekanayake at May 14, 2012 08:46 PM


Waruna Ranasinghe—Webinar - Using WSO2 Carbon to Help Convert One-Sixth of New Zealand’s Population into Mobile Telecom Customers
The webinar for my previous post - WSO2 Middleware to Enable Mobile Services for Telecommunications is scheduled on 17th May 2012. 

Join Asanka Abeysinghe, WSO2 Director Solutions Architecture, and Waruna  Ranasinghe, WSO2 Senior Software Engineer, along with Guest Speaker  Neeraj Satja, 2degrees Mobile Software Development Manager, for a  discussion on how the WSO2 Carbon enterprise middleware platform has  enabled 2degrees to deliver the innovation, high volume and high  performance required to be a mobile market leader.

Click here to register. 


by Waruna at May 14, 2012 04:27 PM

May 13, 2012


Saliya Ekanayake—Taming Wild Horses: Chapel Asynchronous Tasks
Chapel supports nesting data parallel and task parallel code arbitrary as desired. This allows you, for example, to spawn asynchronous tasks inside a forall loop. The code snippet below shows code for a case like this where a forall is run on an array of 3 elements. The work to be done for second element is time consuming, hence a new task is spawned to run the timeeater(). Seems straightforward isn’t it? What if timeeater() takes more time than the forall loop? You’d expect forall to wait till all the dynamically spawned tasks to complete, but unfortunately it’s not the case. So if you want everything to be done when you exit forall loop use the construct sync to synchronize.
 Try running the code with and without sync and observe the value of result, which should be 500500 if forall exit only after all the tasks have completed.
 var d : domain(1) = [1..3];
var a : [d] int = d;
var result : int;
sync forall i in a{
    writeln(i);
    if (i == 2) then {
       begin result = timeeater();
    }
}
writeln("end of forall");
writeln("result ", result);
 proc timeeater(){
    var tmp : int = 0;
    for i in 1 .. 1000{
        tmp = tmp + i;
        if (i%25 == 0) then {
            writeln("eating time ", i);
        }
    }
    return tmp;
}
  
by Saliya Ekanayake at May 13, 2012 06:31 PM


Chris Haddad—PaaS TCO and PaaS ROI: Multi-tenant, shared container PaaS
When investing in technology infrastructure, organizations commonly desire a positive return on investment (ROI) within six to twelve months and a lower PaaS TCO over the investment lifespan.  Does deployment topology sizing, tenant count, tenant density, and service mix significantly impact expense and influence ROI timeframe?

Download the PaaS TCO worksheet to determine multi-tenant, shared container value.
WSO2 clients are considering running middleware as a service instead of deploying traditional middleware silos.  For example, running ESB-as-a-Service across multiple tenants.  The clients are interested in potential cost savings and vendor alternatives.  As mentioned in the ‘Searching for Cloud Architecture’ and PaaS Evaluation Framework blog posts, most PaaS environments, while delivering an application platform as a service, require a dedicated resource pool per tenant application.  For example, a tenant application hosted on Jelastic PaaS will be bound to a set number of Tomcat servers.  Server container level tenancy (e.g. Jelastic, Amazon Beanstalk, RedHat OpenShift, CloudBees) contrasts with shared container tenancy.  In shared container tenancy, tenant applications share a common pool of Java server containers.  The Java server container is dynamically shared across multiple tenant applications.  In single tenant, dedicated container PaaS, significantly more PaaS TCO expense is required to run a PaaS environment compared with a multi-tenant, shared application container PaaS.
The proposed PaaS cost evaluation tool compares multi-tenant, shared application container PaaS TCO with single tenant, dedicated container PaaS TCO (i.e. traditional application server deployment in Cloud) across multiple tenant counts and application platform service combinations.
The worksheet incorporates application platform license (or subscription) cost, PaaS Management service cost, infrastructure expense, and IT management overhead.  Across all scenarios, the worksheet calculates cost when application platforms are deployed on Infrastructure as a Service (IaaS).
Since both a shared application container PaaS and single tenant PaaS can scale up/down on-demand, the worksheet excludes savings derived from on-demand provisioning and teardown.   Instead, the worksheet calculates expense based on the maximum number of JVM instances required for steady-state workloads.
The worksheet does not depict intangible savings derived from faster time to market.  Since the scenarios assume traditional web application architecture, container-level isolation via OSGI, and application clustering (instead of stateless message passing), application migration cost and development cost will be equal and are excluded from the calculation.
Cost Calculation Methodology
The worksheet reflects the following cost components:

Application Platform Subscription (or license fee)
PaaS Management Services (i.e. controllers, load balancer, metering, billing, and monitoring)
Infrastructure as a Service (compute only)
IT Administrative and Management

The Application Platform subscription component calculates the license, vendor maintenance, or subscription expense required to license access to application platform components.   The private Cloud scenarios listed in the worksheet utilize a straight yearly subscription fee per JVM instance.
PaaS Management Services load balance tenants across application platform nodes, monitor PaaS node health, collect metering information, and provision tenant environments.  As many ‘Private PaaS’ deployments replicate traditional application platform topology within the run-time environment, the worksheet removes PaaS Management Service cost from the traditional deployment template scenarios.  If your single tenant, dedicated deployment requires run-time management services (e.g. Cloud Foundry), adjust the worksheet accordingly.  The worksheet may understate the cost required to deploy Cloud Foundry and other single dedicated PaaS delivering on-demand, tenant-aware elastic load balancing.
Across all scenarios, the worksheet calculates cost when application platforms are deployed on Infrastructure as a Service (IaaS).  The worksheet IaaS compute benchmark cost from Amazon AWS EC2 calculator (as of 11 May 2012).  The projected spend is $659 per month for a High-Memory Double Extra Large Instance.
The IT Administrative and Management yearly cost item incorporates administration effort required to generate monthly chargeback/showback statements, configure tenants, monitor run-time environment per tenant, and up-front training investment.
Column B reflects the multi-tenant, shared container scenario.  Column C reflects a single tenant deployment, which is used as input to traditional deployment scenarios (Column D-G).  The traditional deployment scenarios replicate single, dedicated tenant silos in a PaaS environment.   A dedicated tenant silo shares IaaS across JVMs, and dedicates running JVM instances to specific tenants.
Calculation Variables
The following variables may be changed to evaluate cost within your specific context:

Number of tenants
Tenant density per JVM [applied per Application Platform Service]
License instances (i.e. Java Virtual Machine instances or servers) per Application Platform Service and PaaS Management Service
Java Virtual Machine (JVM) density per IaaS Node
Cost per IaaS Node

IaaS compute instance size (e.g. small, medium, large) and application platform footprint (i.e. 256MB, 2GB, 4GB) will influence JVM density per IaaS node.  The worksheet assumes a 32GB IaaS compute instance node with 4GB allocated to each Java Virtual Machine.  The worksheet projects an expected JVM density of 8 JVMs per IaaS node.  The worksheet user may adjust the number of JVMs per IaaS node.
Tenant isolation, partitioning strategies, and service performance will influence tenant density per JVM.  The worksheet’s projected Application Platform Service JVM count considers redundancy, failover, expected load, throughput, and service performance.  The JVM count will typically vary based on hosted Application Platform Service type, throughput, and expected load.  For example, an Identity Management service can often accommodate a greater number of tenants compared with an Enterprise Service bus service.   The tenant density per JVM will vary based on vendor product and workload profile.  In the single tenant scenarios, a single tenant is mapped per dedicated JVM or JVM cluster.  In the multi-tenant shared container scenario, a JVM instance uses OSGI partitioning, tenant specific resource context, and on-demand tenant artifact loading to share JVM resources while maintaining Quality of Service, performance, and security.   In a multi-tenant shared JVM scenario, the worksheet specifies a maximum of seven (7) tenants per JVM in high load situations (e.g. ESB, Application Server, Business Process Management).
Tenant density significantly impacts the number of IaaS nodes and licensed application platform servers.  In the default worksheet configuration, a single tenant per JVM yields a single tenant density of eight (8) tenants per IaaS node.  In a multi-tenant shared JVM scenario, the worksheet specifies a maximum of seven (7) tenants per JVM and 42 tenants per IaaS node.
The worksheet uses a straight subscription support fee per JVM instance and IaaS compute benchmark cost from Amazon AWS EC2 calculator (as of 11 May 2012).  Worksheet users should adjust this number to reflect vendor specific license expense, maintenance expense, subscription investment amounts, and internal cost to deliver IaaS compute nodes.  As storage, network I/O, and external IP addresses will remain constant across all scenarios, the worksheet excludes these costs.
Modifying Cost Scenarios
To modify the cost scenarios and accurately model your expected cost savings from multi-tenant, shared container PaaS, you should collect the following load information:

A specified number of tenant projects in Year 1,2,3
Known tenant load (e.g. web requests, transactions, memory, CPU, latency)
Known mapping between tenant load and service capacity

The information can be used to derive the following variable values:

Expected number of tenants
Number of Application Platform Service JVMs
JVM density per IaaS node

Modify the worksheet to reflect your specific environment context. Cost variables are shown in ‘blue font’.
Bottom Line
In large-scale, multiple application service deployments (i.e. 100 tenants and 5 services), multi-tenant shared container PaaS is 14 times (14X, 1400%) more efficient than single container deployment when measuring number of JVM instances.  Positive financial TCO is achieved after sixteen (16) tenants subscribe to the environment.  After all 100 tenants have subscribed, the multi-tenant shared container PaaS is 659% more cost efficient than single container deployments. Table 1 illustrates the details.
In large-scale, single application service deployments (i.e. 100 tenants and 1 services), multi-tenant shared container PaaS is 13 times (13X, 1300%) more efficient than single container deployment when measuring number of JVM instances.  Positive financial TCO is achieved after twenty-two (22) tenants subscribe to the environment.  After all 100 tenants have subscribed, the multi-tenant shared container PaaS is 475% more cost efficient than single container deployments.  Table 2 illustrates the details.
In small-scale, multiple application service deployments (i.e. 8 tenants and 2 services), multi-tenant shared container PaaS is three times (3x, 300%) more efficient than single container deployment when measuring number of JVM instances. Positive financial TCO is achieved after four (4) tenants subscribe to the environment.  After all eight (8) tenants have subscribed, the multi-tenant shared container PaaS is 214% more cost efficient than single container deployments.  Table 3 illustrates the details.
In small-scale, single application service deployments (i.e. 8 tenants and 1 service), multi-tenant shared container PaaS is two times (2.2x, 220%) more efficient than single container deployment when measuring number of JVM instances. Positive financial TCO is achieved after five (5) tenants subscribe to the environment.  After all eight (8) tenants have subscribed, the multi-tenant shared container PaaS is 166% more cost efficient than single container deployments.  Table 4 illustrates the details.
 
Table 1: High scale, multiple application service deployment




On-Premise PaaS vs. On-Premise Hosting


Shared Container PaaS Scenario


Traditional Deployment Template


Traditional – Scenario 1


Traditional – Scenario 2


Traditional – Scenario 3


Traditional – Scenario 4




Tenants and Partitioning









Tenant Isolation Level

Shared PaaS Nodes


Dedicated Instances


Dedicated Instances


Dedicated Instances


Dedicated Instances


Dedicated Instances



Tenants

100


1


15


16


50


100




PaaS Management Services [# of JVMs]


11


0


0


0


0


0



PaaS Controller and Registry

3


0


0


0


0


0



Load Balancer for management traffic

3


0


0


0


0


0



Cloud Manager (provisioning, metering, billing)

1


0


0


0


0


0



Business Activity Monitor and summarizer

4


0


0


0


0


0




Application Platform Services [# of JVMs]









Application Server

14


2


30


32


100


200



Gadget Server

0


0


0


0


0


0



Mashup Server

0


0


0


0


0


0



Enterprise Service Bus

14


2


30


32


100


200



Governance Registry

2


1


15


16


50


100



Identity Server

2


1


15


16


50


100



Business Process Management

0


0


0


0


0


0



Business Rules

0


0


0


0


0


0



Complex Event Processing Server

0


0


0


0


0


0



Data Services Server

4


1


15


16


50


100



Message Broker

0


0


0


0


0


0



Cassandra Storage Service

0


0


0


0


0


0



Relational Storage Service

0


0


0


0


0


0



File Storage Service

0


0


0


0


0


0



API Management

0


0


0


0


0


0



Business Activity Monitoring

0


0


0


0


0


0



Load Balancer for Application Platform Service Clusters

3


0


0


0


0


0




Platform Footprint and Cost









Number of JVM Instances

50


7


105


112


350


700



Cost Per JVM Instance

$8,000


$8,000


$8,000


$8,000


$8,000


$8,000



Total Platform Subscription Cost

$400,000


$56,000


$840,000


$896,000


$2,800,000


$5,600,000



Platform Subscription Cost Comparison (App Platform)

$0


($344,000)


$440,000


$496,000


$2,400,000


$5,544,000




Infrastructure as a Service









JVM Density per IaaS node (32GB IaaS, 4GB JVM)

8


8


8


8


8


8



Calculated number of IaaS nodes

7


1


14


14


44


88



Cost per IaaS Year

$7,908


$7,908


$7,908


$7,908


$7,908


$7,908



Total IaaS Cost

$55,356


$7,908


$110,712


$110,712


$347,952


$695,904



Platform Subscription Cost Comparison (IaaS)

$0


($47,448)


$55,356


$55,356


$292,596


$640,548




IT Administrative and Management Cost









Chargeback/showback billing

12000


60


900


960


3000


6000



Tenant configuration

610


60


870


900


2820


5640



Tenant monitoring and systems administration

2600


364


5460


5824


18200


36400



Cost per IT man hour

$60


$60


$60


$60


$60


$60



WSO2 Stratos Platform Training Cost

$25,000


$0


$0


$0


$0


$0



Total IT Administrative and Management Cost

$937,600


$29,040


$433,800


$461,040


$1,441,200


$2,882,400



Platform Subscription Cost Comparison (IT Admin & Mgt)

$0


($908,560)


($503,800)


($476,560)


$503,600


$1,944,800




Total Cost of Ownership









Total Cost

$1,392,956


$92,948


$1,384,512


$1,467,752


$4,589,152


$9,178,304



Total Savings / (Extra Expense)

$0


($1,300,008)


($8,444)


$74,796


$3,196,196


$7,785,348



JVM Instance Efficiency


0.1


2.1


2.2


7.0


14.0



Cost Efficiency


7%


99%


105%


329%


659%




 
 
Table 2: High scale, single application service deployment




On-Premise PaaS vs. On-Premise Hosting


Shared Container PaaS Scenario


Traditional Deployment Template


Traditional  Deployment


Traditional  Deployment


Traditional  Deployment


Traditional  Deployment




Tenants and Partitioning









Tenant Isolation Level

Shared PaaS Nodes


Dedicated Instances


Dedicated Instances


Dedicated Instances


Dedicated Instances


Dedicated Instances



Tenants

100


1


10


22


50


100




PaaS Management Services [# of JVMs]


10


0


0


0


0


0



PaaS Controller and Registry

3


0


0


0


0


0



Load Balancer for management traffic

2


0


0


0


0


0



Cloud Manager (provisioning, metering, billing)

1


0


0


0


0


0



Business Activity Monitor and summarizer

4


0


0


0


0


0




Application Platform Services [# of JVMs]









Application Server

0


0


0


0


0


0



Gadget Server

0


0


0


0


0


0



Mashup Server

0


0


0


0


0


0



Enterprise Service Bus

14


2


20


44


100


200



Governance Registry

2


1


10


22


50


100



Identity Server

1


1


10


22


50


100



Business Process Management

0


0


0


0


0


0



Business Rules

0


0


0


0


0


0



Complex Event Processing Server

0


0


0


0


0


0



Data Services Server

0


0


0


0


0


0



Message Broker

0


0


0


0


0


0



Cassandra Storage Service

0


0


0


0


0


0



Relational Storage Service

0


0


0


0


0


0



File Storage Service

0


0


0


0


0


0



API Management

0


0


0


0


0


0



Business Activity Monitoring

0


0


0


0


0


0



Load Balancer for Service Clusters

3


0


0


0


0


0




Platform Footprint and Cost









Number of JVM Instances

30


4


40


88


200


400



Cost Per JVM Instance

$8,000


$8,000


$8,000


$8,000


$8,000


$8,000



Total Platform Subscription Cost

$240,000


$32,000


$320,000


$704,000


$1,600,000


$3,200,000



Platform Subscription Cost Comparison (App Platform)

$0


($208,000)


$80,000


$464,000


$1,360,000


$3,168,000




Infrastructure as a Service









JVM Density per IaaS node (32GB IaaS, 4GB JVM)

8


8


8


8


8


8



Calculated number of IaaS nodes

4


1


5


11


25


50



Cost per IaaS Year

$7,908


$7,908


$7,908


$7,908


$7,908


$7,908



Total IaaS Cost

$31,632


$7,908


$39,540


$86,988


$197,700


$395,400



Platform Subscription Cost Comparison (IaaS)

$0


($23,724)


$7,908


$55,356


$166,068


$363,768




IT Administrative and Management Cost









Chargeback/showback billing

12000


60


600


1320


3000


6000



Tenant configuration

440


48


330


726


1650


3300



Tenant monitoring and systems administration

1560


208


2080


4576


10400


20800



Cost per IT man hour

$60


$60


$60


$60


$60


$60



WSO2 Stratos Platform Training Cost

$25,000


$0


$0


$0


$0


$0



Total IT Administrative and Management Cost

$865,000


$18,960


$180,600


$397,320


$903,000


$1,806,000



Platform Subscription Cost Comparison (IT Admin & Mgt)

$0


($846,040)


($684,400)


($467,680)


$38,000


$941,000




Total Cost of Ownership









Total Cost

$1,136,632


$58,868


$540,140


$1,188,308


$2,700,700


$5,401,400



Total Savings / (Extra Expense)

$0 


($1,077,764)



($596,492)



$51,676 


$1,564,068 


$4,264,768 



JVM Instance Efficiency


0.1


1.3


2.9


6.7


13.3



Cost Efficiency


5%


48%


105%


238%


475%




 
 
Table 3: Small scale, multiple application service deployment




On-Premise Shared PaaS vs. On-Premise Single Tenant Hosting


Shared Container PaaS Scenario


Traditional Deployment Template


Traditional – Scenario 1


Traditional – Scenario 2


Traditional – Scenario 3


Traditional – Scenario 4




Tenants and Partitioning









Tenant Isolation Level

Shared PaaS Nodes


Dedicated Instances


Dedicated Instances


Dedicated Instances


Dedicated Instances


Dedicated Instances



Tenants

8


1


2


4


6


8




PaaS Management Services [# of JVMs]


8


0


0


0


0


0



PaaS Controller and Registry

3


0


0


0


0


0



Load Balancer for management traffic

2


0


0


0


0


0



Cloud Manager (provisioning, metering, billing)

1


0


0


0


0


0



Business Activity Monitor and summarizer

2


0


0


0


0


0




Application Platform Services [# of JVMs]









Application Server

2


2


4


8


12


16



Gadget Server

0


0


0


0


0


0



Mashup Server

0


0


0


0


0


0



Enterprise Service Bus

2


2


4


8


12


16



Governance Registry

2


1


2


4


6


8



Identity Server

1


1


2


4


6


8



Business Process Management

0


0


0


0


0


0



Business Rules

0


0


0


0


0


0



Complex Event Processing Server

0


0


0


0


0


0



Data Services Server

1


1


2


4


6


8



Message Broker

0


0


0


0


0


0



Cassandra Storage Service

0


0


0


0


0


0



Relational Storage Service

0


0


0


0


0


0



File Storage Service

0


0


0


0


0


0



API Management

0


0


0


0


0


0



Business Activity Monitoring

0


0


0


0


0


0



Load Balancer for Application Platform Service Clusters

2


0


0


0


0


0




Platform Footprint and Cost









Number of JVM Instances

18


7


14


28


42


56



Subscription cost per JVM Instance [per year]

$8,000


$8,000


$8,000


$8,000


$8,000


$8,000



Total Platform Subscription Cost [per year]

$144,000


$56,000


$112,000


$224,000


$336,000


$448,000



Platform Subscription Cost Comparison (App Platform)

$0


($88,000)


($32,000)


$80,000


$192,000


$392,000




Infrastructure as a Service









JVM Density per IaaS node (32GB IaaS, 4GB JVM)

8


8


8


8


8


8



Calculated number of IaaS nodes

3


1


2


4


6


7



Cost per IaaS node [per year]

$7,908


$7,908


$7,908


$7,908


$7,908


$7,908



Total IaaS Cost [per year]

$23,724


$7,908


$15,816


$31,632


$47,448


$55,356



Platform Subscription Cost Comparison (IaaS)

$0


($15,816)


($7,908)


$7,908


$23,724


$31,632




IT Administrative and Management Cost









Chargeback/showback billing

960


60


120


240


360


480



Tenant configuration

178


60


120


240


360


450



Tenant monitoring and systems administration

936


364


728


1456


2184


2912



Cost per IT man hour

$60


$60


$60


$60


$60


$60



WSO2 Stratos Platform Training Cost

$25,000


$0


$0


$0


$0


$0



Total IT Administrative and Management Cost [per year]

$149,440


$29,040


$58,080


$116,160


$174,240


$230,520



Platform Subscription Cost Comparison (IT Admin & Mgt)

$0


($120,400)


($91,360)


($33,280)


$24,800


$81,080




Total Cost of Ownership









Total Cost [Year 1]

$317,164


$92,948


$185,896


$371,792


$557,688


$733,876



Total Savings for PaaS / (Extra Expense)

$0 


($224,216)



($131,268)



$54,628 


$240,524 


$416,712 



JVM Instance Efficiency


0.4


0.8


1.6


2.3


3.1



Cost Efficiency


29%


59%


117%


176%


231%




 
 
Table 4: Small scale, single application service deployment




On-Premise PaaS vs. On-Premise Hosting


Shared Container PaaS Scenario


Traditional Deployment Template


Traditional – Scenario 1


Traditional – Scenario 2


Traditional – Scenario 3


Traditional – Scenario 4




Tenants and Partitioning









Tenant Isolation Level

Shared PaaS Nodes


Dedicated Instances


Dedicated Instances


Dedicated Instances


Dedicated Instances


Dedicated Instances



Tenants

8


1


2


4


5


8




PaaS Management Services [# of JVMs]


7


0


0


0


0


0



PaaS Controller and Registry

2


0


0


0


0


0



Load Balancer for management traffic

2


0


0


0


0


0



Cloud Manager (provisioning, metering, billing)

1


0


0


0


0


0



Business Activity Monitor and summarizer

2


0


0


0


0


0




Application Platform Services [# of JVMs]









Application Server

0


0


0


0


0


0



Gadget Server

0


0


0


0


0


0



Mashup Server

0


0


0


0


0


0



Enterprise Service Bus

2


2


4


8


10


16



Governance Registry

1


1


2


4


5


8



Identity Server

0


0


0


0


0


0



Business Process Management

0


0


0


0


0


0



Business Rules

0


0


0


0


0


0



Complex Event Processing Server

0


0


0


0


0


0



Data Services Server

0


0


0


0


0


0



Message Broker

0


0


0


0


0


0



Cassandra Storage Service

0


0


0


0


0


0



Relational Storage Service

0


0


0


0


0


0



File Storage Service

0


0


0


0


0


0



API Management

0


0


0


0


0


0



Business Activity Monitoring

0


0


0


0


0


0



Load Balancer for Application Platform Service Clusters

1


0


0


0


0


0




Platform Footprint and Cost









Number of JVM Instances

11


3


6


12


15


24



Subscription cost per JVM Instance [per year]

$8,000


$8,000


$8,000


$8,000


$8,000


$8,000



Total Platform Subscription Cost [per year]

$88,000


$24,000


$48,000


$96,000


$120,000


$192,000



Platform Subscription Cost Comparison (App Platform)

$0


($64,000)


($40,000)


$8,000


$32,000


$168,000




Infrastructure as a Service









JVM Density per IaaS node (32GB IaaS, 4GB JVM)

8


8


8


8


8


8



Calculated number of IaaS nodes

2


1


1


2


2


3



Cost per IaaS node [per year]

$7,908


$7,908


$7,908


$7,908


$7,908


$7,908



Total IaaS Cost [per year]

$15,816


$7,908


$7,908


$15,816


$15,816


$23,724



Platform Subscription Cost Comparison (IaaS)

$0


($7,908)


($7,908)


$0


$0


$7,908




IT Administrative and Management Cost









Chargeback/showback billing

480


60


120


240


300


480



Tenant configuration

120


44


58


116


130


202



Tenant monitoring and systems administration

572


156


312


624


780


1248



Cost per IT man hour

$60


$60


$60


$60


$60


$60



WSO2 Stratos Platform Training Cost

$25,000


$0


$0


$0


$0


$0



Total IT Administrative and Management Cost [per year]

$95,320


$15,600


$29,400


$58,800


$72,600


$115,800



Platform Subscription Cost Comparison (IT Admin & Mgt)

$0


($79,720)


($65,920)


($36,520)


($22,720)


$20,480




Total Cost of Ownership









Total Cost [Year 1]

$199,136


$47,508


$85,308


$170,616


$208,416


$331,524



Total Savings for PaaS / (Extra Expense)

$0 


($151,628)



($113,828)



($28,520)



$9,280 


$132,388 



JVM Instance Efficiency


0.3


0.5


1.1


1.4


2.2



Cost Efficiency


24%


43%


86%


105%


166%




 
 

by haddadcblg at May 13, 2012 01:27 PM


Isuru Suriarachchi—4 years at WSO2
Yesterday morning, accidentally I realized that I’ve been with WSO2 for 4 years. Joined the company on the 12th of May 2008 just after completing my CSE degree. Supun, Milinda, Saliya, Kalani and Rajika were the other batch mates who joined with me and Sameera joined a week later. Looking back, it has been a wonderful period of my life. I’ve learned a lot, gathered so much experience specially at customer sites and made lots of friends.
Just after joining the company I was assigned into WSO2 WSAS team and Azeez was my very first product manager. I still remember how we worked for our very first Carbon release. It was my first release experience and we had to put in lot of effort to get the release out. However I never felt tired and it was fun. I’ve written this post on the 31st of December 2008 with all my feelings about the company and the start of my career.
In addition to the technical experiences that I’ve gathered, I’ve made lots of friends at WSO2 who contributed a lot to make these 4 years unforgettable. Specially the annual “Adyapana Charikawa”   organized by Charitha, has added loads of fun memories. In addition to that I always enjoyed playing Carrom, Table Tennis and Basket Ball with our guys whenever we get a chance.
Having spent such a wonderful time, most probably I’ll be leaving the company for my studies in August. It’s little sad to think about leaving all my WSO2 friends. But still I don’t think about that too much as I’ve got 3 more months to enjoy with them  .

         
by isurues at May 13, 2012 08:07 AM

May 12, 2012


Ganesh Prasad—Tight Coupling in the TCP/IP Stack!

Scandal and embarrassment!

The TCP/IP stack of Internet protocols, the poster child for a layered architecture with well-defined responsibilities and interfaces that abstract out needless dependencies, has a dirty little secret that I just stumbled upon since I now work for a telco. But before I tell you what it is, a quick recap of how the technology works.

IP stands for Internet Protocol. Every device has an IP address. [IP version 4 has been the most common so far, and IPv4 addresses look like this: 192.168.1.1 (or in hex, C0.A8.01.01). IP version 6 (IPv6)  addresses look like this: 3ffe:1900:4545:3:200:f8ff:fe21:67cf.]

The way the Internet works is by routing packets of information, hop-by-hop, from a source to a destination. Each node along the way knows, by looking at the destination IP address in a packet, how to forward that packet so it gets one step closer to its destination. So all that an IP network really has are routing smarts. It's the destination's IP address in a message that holds all the information required for it to reach its intended audience.

That's great when the sources and destinations of messages are fixed in location. They have a certain IP address assigned to them when they start up, and from then on, that IP address typically doesn't change until they next start up.

Mobile data devices (which include 3G mobile phones and later devices that use the packet-switched data network) have introduced a problem. Their IP addresses need to keep changing because they connect to different nodes (or cells, or towers) as they move, and it would play havoc with routing if they carried their original IP addresses around when connecting to new nodes. So fine, the technology allows for their IP addresses to change dynamically. However, the logical data connections that the devices establish need to remain for the duration of the session. There could be a download going on, for example, and an interruption of the connection will abort the download. Innovations such as "fast mobile IP" were introduced to mitigate the visible effects of the problem, but did not address its root cause.

The root cause lies in a rather ugly fact about IP addresses. An IP address confuses a device's identity with its location. A device's location keeps changing as it moves, but its identity does not change. A location is important to know where packets are to be delivered. But logical concepts like sessions need to be tied to a device's identity, not to its location. These are two different concepts, but a single mechanism (the IP address) has been chosen to implement both of them. As long as the location and identity did not independently change, the design flaw remained hidden. Now with data-enabled mobile devices, device location and device identity show themselves very clearly as two different things, and the conceptual limitation of the IP address has therefore been exposed.

That's the rationale behind the new protocol specification called HIP (Host Identity Protocol). HIP is meant to sit between TCP and IP. Normally, a TCP-level domain name is resolved by DNS to an IP address. A whole generation of IT professionals has come of age with this principle internalised as an axiom of How Things Work. HIP is a Copernicus or a Galileo challenging an established view. The Sun does not go round the Earth, after all. It's the Earth that goes round the Sun! That's going to take some getting used to. For a networking professional or a web architect, discovering that the venerable TCP/IP stack should actually be the TCP/HIP/IP stack is a bit like discovering that they're an adopted child. But however painful the realisation and readjustment, it's better that the truth be known.

Under the new proposal, a TCP-level domain name needs to be resolved by DNS to a logical HIP name, which then gets further resolved to an IP address! Now, if a device is moving, its IP address can keep changing, but its HIP name will remain the same. Therefore TCP connections need not be torn down and re-established. Sessions need not be dropped and re-created.

RFC 4423 (HIP Architecture) says:
In the current Internet, the transport layers are coupled to the IP addresses. Neither can evolve separately from the other.
[...]
There are three critical deficiencies with the current namespaces. First, dynamic readdressing cannot be directly managed.  Second, anonymity is not provided in a consistent, trustable manner. Finally, authentication for systems and datagrams is not provided. All of these deficiencies arise because computing platforms are not well named with the current namespaces. 
It goes on to say:
An independent namespace for computing platforms could be used in end-to-end operations independent of the evolution of the internetworking layer and across the many internetworking layers. This could support rapid readdressing of the internetworking layer because of mobility, rehoming, or renumbering. 
Amazing, isn't it? We've been nursing a tightly-coupled serpent in our collective bosom for over 3 decades, and we didn't even know...

It's going to take a while for HIP to become part of the Internet ecosystem (if it ever will!) The power of entrenched ways of thinking could prove too powerful to allow a much-needed rationalisation.

The lesson for me personally is that if we don't architect a system right, we will live with its negative implications for a long, long time. Even the founding fathers of the Internet, geniuses as they were, were not perfect, and we can clearly see in hindsight how a conceptual blunder (a conflation of location with identity) has impacted us.

I do believe though, that even the current HIP proposal is making a blunder of its own by confusing identifiers with identity credentials. RFC 4423 says:

In theory, any name that can claim to be 'statistically globally unique' may serve as a Host Identifier.  However, in the authors' opinion, a public key of a 'public key pair' makes the best Host Identifier.  As will be specified in the Host Identity Protocol specification, a public-key-based HI can authenticate the HIP packets and protect them from man-in-the-middle attacks. 
From my own work on Identity Management, I have come to realise that multiple sets of credentials can be used to arrive at, or establish, an identity. The establishment of an identity within a given context requires an identifier. This identifier may be the credentials themselves, or something else. It's important to realise that the "may be" should not be taken as a "must be". For the purpose of security, the authors of the HIP specification are proposing that verifiable credentials be used as the identifier in all situations. I fear that will result in a similar problem later on when the requirements of authentication and identity establishment diverge in some context. I'll write to the committee explaining my concerns.

by Ganesh Prasad at May 12, 2012 04:43 PM


Saliya Ekanayake—Chapel is Sweet
It has been a little while since I started playing around Chapel (http://chapel.cray.com/) language, but could not run anything fun and large until recently. As part of the B524 – Parallelism in Programming Languages and Systems class from Prof. Lumsdaine (http://osl.iu.edu/~lums/), we had to implement Single Source Shortest Path (SSSP) of Graph500 (http://www.cc.gatech.edu/~jriedy/tmp/graph500/) specification. Only then I could realize the easiness of many of the high-level abstractions provided in Chapel compared to other parallel languages or paradigms. Honestly, I did not expect it to work in the first run across a set of machines, but surprisingly it did!
  
by Saliya Ekanayake at May 12, 2012 02:34 PM


Saliya Ekanayake—Download a Set of URLs with GNU Wget
I had a list of URLs that I wanted to download and it was a pain to do it manually. So end up writing a simple shell script and downloading all of them using GNU Wget. Here’s the shell script (modified the one at http://www.linuxquestions.org/questions/programming-9/shell-script-that-read-each-line-separatly-364259/).
#!/bin/bash
# Set the field seperator to a newline
IFS="
"
# Loop through the file
for line in `cat file.txt`;do
 wget $line
done
  
by Saliya Ekanayake at May 12, 2012 02:17 PM

May 10, 2012


Tharindu Mathew—Do you trust Google Big Query with your Big Data?
Google has come up with a fantastic service to analyze large amounts of data. It’s called BigQuery and it allows you to run analysis on big data on the cloud. As expected, the tool has a superb, intuitive web UI. The data analysis language uses SQL like queries. (Hive, anyone   ). Have a look at the  Big Query Tutorial, it looks pretty neat. So, now all you need to do to run queries is to upload your data to Google using the form shown below.  It allows you to upload a file or point to it using Google’s cloud storage.
Now, the interesting question here is that to analyze using BigQuery how much of that data are you willing to give Google? And how long will that take? The answer won’t be “Let me quickly upload a 500 GB file and run some queries”. That amount of data would definitely take some time to upload. So, effectively, this SaaS becomes pretty useless as more and more data volumes need to be uploaded for analysis.
Everyone trusts Google (   ), so this concern might be easily ignored. But a potential other problem I see is the “Privacy Policies” that are violated. Usually, when you want to analyze data, it can contain sensitive data such as user behavior patterns and so forth. How comfortable will your customers be if you hand that data over to Google? Even anonymizing this data might not save you from a potential legal breach.
I still believe setting up your own data analysis and monitoring platform is the best way to go. Thoughts? I’d love to hear them.

         
by Mackie Mathew at May 10, 2012 11:59 AM

May 09, 2012


Ganesh Prasad—The Unholy Alliance of Analyst and Big Vendor

I was browsing IT news as usual and came across a commentary page that discussed Progress Software's recent decision to divest itself of several core products, including Sonic ESB, Savvion BPM and Actional Service Management. To tell you the truth, I wasn't paying attention to the website where this page was hosted. But then I came upon this statement:

The situation reminds us of a key benefit of selecting a top-tier enterprise vendor: IBM and SAP rarely kill established products, and the same can be said of Oracle in recent years.

I've been in the industry long enough to detect a "this-message-brought-to-you-by-your-friendly-neighbourhood-commercial-sponsor" marketing insert when I see one. So I glanced up at the website, and sure enough, it was one of the big analysts, Forrester to be precise.

The funny thing is, I would have drawn a very different inference from the news about Progress.

This news confirms a few things I already know:

1. Commoditisation continues its onward march in product category after product category. This is good news for customers because it means lower prices as well as standardisation of features (which in turn leads to greater interoperability and lower operating costs).

2. Commoditisation is dreaded by vendors because it erodes their profit margins. The large vendors have enough of a lock on their customers through network externalities that they can often maintain their profit margins in spite of commoditisation. But smaller commercial vendors must either exit these market segments or accept lower margins.

3. The unspoken trend in any modern-day commoditisation story is the rise of Open Source. After all, Progress is not quitting these middleware markets because of competition from the big vendors. The pressure on them is from below, from Open Source. And there is a thriving market here in the "supported Open Source" category, as this gleeful blog post from WSO2's VP of Technology Evangelism shows.

I'm disappointed (but not surprised) that a big analyst is spinning what is really a welcome story of commoditisation into a warning to customers to buy the big expensive brands, or else. I guess there's no money in it for them to recommend that customers choose Open Source alternatives to start with.

But really, chee, how low can they go?


by Ganesh Prasad at May 09, 2012 01:13 PM


Ganesh Prasad—Mentoring NoSQL From Adolescence to Maturity

NoSQL is more than just the flavour of the month. There's no doubt that it's here to stay. But the movement is now experiencing growing pains.

In short, most variants of NoSQL have established a niche for themselves by creatively dropping one or more of the ACID constraints of traditional relational databases. Consequently, what they've gained on the swings in terms of features, they've lost on the roundabouts. Today, some of those shortcomings are becoming pain points, and the respective projects are attempting to layer the missing features on top of their existing products.

An expert view is that this is the wrong approach to take. Databases are complex beasts, and their features cannot be layered on, but engineered concurrently. It's a non-intuitive insight, but one that the greybeards of the industry have learnt through hard experience over decades.

It's very timely that one of the most respected names in the database field, IBM Fellow C Mohan, has stepped up to provide much-needed leadership and guidance to the NoSQL movement. His initial analysis and critique of NoSQL is on his blog (Part 1, Part 2, Part 3 and Part 4).

Mohan has promised to study individual NoSQL databases in more detail so as to understand their design nuances better. If he can then propose ways for these projects to enhance their capabilities in the most effective way, he would have succeeded in enabling a whole new wave of web applications.

Here's wishing him all success!

by Ganesh Prasad at May 09, 2012 01:10 PM


Afkham Azeez—Lazy Loading Webapps in a PaaS Deployment




In Lazy Loading Deployment Artifacts in a PaaS Deployments posted last November, I talked about the principles behind lazy loading & the advantages of lazy loading for a PaaS deployment. I concluded the article with the plans we had for implementing lazy loading for webapps. The WSO2 AppServer team has just completed this feature and carried out a performance comparison.




The following graph shows the behavior of the initial response time when the number of deployed webapps increase, with & without ghost deployment (GD). We started with 1 webapp, and gradually increased the number of deployed webapps up to 300. As you can see, without ghost deployment, the initial response time steadily increases, eventually resulting in client timeouts.





This perhaps is the first time webapp lazy loading has been implemented for Tomcat. WSO2 AppServer uses embedded Tomcat. The lazy loading facility is available for the standalone version of the product as well as the multi-tenanted PaaS version of the WSO2 AppServer.




This implementation also includes unloading of webapps which have not been used for sometime, thereby allowing optimal usage of the available resources.




We have written a Tomcat valve which will check whether the real form of the webapp has been loaded, and load it if necessary, and there is an unloader task which periodically checks for inactive webapps & unloads the real webapp & replaces it with the ghostified form of that webapp.

by Afkham Azeez at May 09, 2012 12:48 PM

May 07, 2012


Hasini Gunasinghe—Setting up cloud manager of Stratos with an external LDAP
Cloud manager is the service in WSO2 Stratos which provides the core functionalities of tenant management like creating new tenants by the super admin and self registration of tenants.

Stratos 1.5 is going to be released with in next couple of weeks. Then you will be able to download the distribution from wso2.org.
For now you need to check out the source from https://svn.wso2.org/repos/wso2/branches/carbon/3.2.0/ and build the stratos manager in products/manager/  location.

You can setup stratos on your machine using the setup script (stratos-setup.pl) which is available here.
In order to run that script, you need to: 
have perl installed in your machine. In Linux, you can install "liblist-moreutils-perl".
Have my-sql installed in the machine and have username and password root/root. 
set the following environment variables in the system.
export CARBON_DIR=/home/hasini/WSO2/branch_3.2.0/carbon
export STRATOS_DIR=/home/hasini/WSO2/stratos testing/1.5.0/setup/unpacks
export STRATOS_VERSION=1.5.0
export SSO_ENABLED=false
export CREATE_DB=true
export STARTUP_DELAY=30 
export PACKS_DIR=/home/hasini/WSO2/stratos testing/1.5.0/setup/packs
export PRODUCT_LIST="manager"

Following is an explanation on each of the above environment variable:
CARBON_DIR - root directory of your carbon check out. You need this if you build the stratos service distributions from source.
STRATOS_DIR - this is the directory to where the distributions zip files should be unpacked by the script.
STRATOS_VERSION - current stratos version
SSO_ENABLED - whether single sign on should be enabled across the different stratos services. In this case, we do not need it.
CREATE_DB - whether stratos related databased in my-sql should be created (if exists and this value is true, tables will be dropped and recreated)
STARTUP_DELAY - if several stratos services are going to be started, the delay between two startups
PACKS_DIR - if you are settting up stratos with downloaded packs, this is the folder where those packs are located.
PRODUCT_LIST - the list of stratos services that should be set up from this script.

Now all set, lets follow the following steps to start the cloud manager with an external ApacheDS LDAP server.
Create a new partition in ApacheDS LDAP where the separate user,group bases for each tenant will be created, when creating tenants through cloud manager. You may refer to my previous blog post for creating a new partition in ApacheDS. You may also create separate user base and group base for super tenant space. (i.e ou=Users,dc=wso2,dc=org & ou=groups,dc=wso2,dc=org)
Go to the above mentioned STRATOS_DIR where stratos manager has been unpacked, and change the following configuration files found in [carbon_home]/repository/conf
tenant-mgt.xml: There you may notice that default tenant manager is JDBCTenantManager, but we now support tenant management with LDAP as well which we are going to demonstrate here. (if you didn't find this file in the above location, create a file named tenant-mgt.xml and copy the content shown below.)
comment out the JDBCTenantManager and uncomment the CommonHybridLDAPTenantManager. 
set the "RootPartition" as the partition name that we created in step 1 above.
Following is the tenant-mgt.xml in our case: 

    dc=wso2,dc=com
    organizationalUnit
    ou
    organizationalUnit
    ou

user-mgt.xml: 
locate the property "MultiTenantRealmConfigBuilder". This is the class which is responsible for cloning the bootstrap user-mgt.xml and creating tenant-specific user-mgt.xml which contains the specific user and group bases of each tenant. Set it as follows:  
org.wso2.carbon.user.core.config.multitenancy.CommonLDAPRealmConfigBuilder
Comment out the JDBCUserStoreManager which comes by default and uncomment the ApacheDSUserStoreManager for an external LDAP in read/write mode. 
In ApacheDSUserStoreManager configuration element, set ConnectionURL, ConnectionName, ConnectionPassword, UserSearchBase and GroupSearchBase for WSO2 user manager to connect to your external LDAP server.

            false
            ldap://localhost:10389
            uid=admin,ou=system
            secret
            SHA
            (objectClass=person)
            inetOrgPerson
            ou=Users,dc=wso2,dc=com
            (&(objectClass=person)(uid=?))
            uid
            [\\S]{5,30}
            true
            true
            false
            ou=Groups,dc=wso2,dc=com
            (objectClass=groupOfNames)
            groupOfNames
            (&(objectClass=groupOfNames)(cn=?))
            cn
            member
        
embedded-ldap.xml:
Disable the embedded-ldap server by setting "enable" property to false.
       3. Start the external ApacheDS server.
       4. Start the WSO2 Stratos Manager by running wso2server.sh/.bat scripts in  [stratos_manager_home] /bin directory.
       5. Access management console through https://localhost:9443/carbon.
       6. Login as super tenant with username: admin, password: secret (note that these are the credentials of admin user that you specified in user-mgt.xml)
       7.  Add a new tenant by providing tenant details as follows:
    8. After the tenant is successfully registered, you need to activate the tenant by checking the "activate" check box.
    9. Then log out from the super admin account.
   10. Login as the tenant admin of newly created tenant, by providing the above given credentials. For an example, in the above case, user name should be hasini@willpower.org.
You can create new users, new groups and assign users to those groups under your tenant.
If you look at the multi-tenanted LDAP structure from ApacheDirectory Studio, you will observer that there is a separate organizational unit is created for the tenant wso2.org and inside that, separate user base and a group base is created as shown in the following diagram.

That's it... You can create more tenants and experience the tenant management with a multi-tenanted LDAP.

by Hasini at May 07, 2012 06:41 AM

May 06, 2012


Afkham Azeez—The role of a Load Balancer in a Platform-as-a-Service



Introduction

Load balancing is a key concern in a Platform-as-a-Service (PaaS) or a middleware platform deployed on the Cloud. In this article, we will look at some key concepts related to load balancing, and the the significance & usage of load balancers in PaaS deployments.




The primary functionality of a load balancer is to distribute the load of the incoming traffic amongst a set of backend worker nodes. This set of worker nodes can be either statically configured or can be dynamically discovered. Traditionally, load balancers are configured with a set of static nodes. This means, new nodes outside this set cannot be added at runtime. Dynamic load balancers support addition & removal of worker nodes at runtime, and the IP addresses & other connection details of the backend nodes need not be known in advance by the load balancer. The load balancing algorithm is a central part of a load balancer. This algorithm specifies the load balancing policy, or how the load has to be distributed across multiple backend worker nodes. Generally, all worker nodes have identical hardware & software configuration, as well as host identical copies of deployment artifacts. Hence the round-robin load balancing algorithm is very suitable & widely used for such deployments.


Session Affinity


Most modern load balancers support session affinity. This means, if the client sends a session ID, the load balancer will forward all requests containing a particular session ID, to the same backend worker node, irrespective of the specified load balancing algorithm. This may look like defeating the purpose of load balancing. But before the session is created, the request will first be dispatched to the worker node which is next in-line, and a session will be established with that worker node. We also have to keep in mind that stateful applications inherently do not scale well, and state replication can have huge overheads, so it is best to minimize server side state if you want your application to be massively scalable. So, session-affinity based load balancing is a compromise solution to the problem of deploying stateful applications in clusters.


Elastic Load Balancer


An Elastic Load Balancer (ELB), in addition to carrying out its functionality in load balancing, is also responsible for monitoring the load & starting up new worker nodes or terminating existing worker nodes, depending on the load. This behavior of scaling up the system while the load increases & scaling down the system when the load decreases is known as autoscaling.



In a typical architecture, load balancing & autoscaling will be handled by two logically distinct components. It may even be possible to deploy the load balancer component & the autoscaler component separately.


Cloud-nativity & Load Balancing


Load balancing is the key to Cloud-based deployment architectures. The Elastic Load Balancer is an essential component in the deployment architecture, when it comes to realizing the Cloud native attributes of multi-tenancy, elasticity, distributed & dynamic wiring, and incremental deployment & testability.


Fronting Multiple Clusters - Service-aware Load Balancing









When it comes to production deployments, a load balancer does not do much of the real work. The real work is done in the backend worker nodes. So, having load balancers introduces additional cost. Since load balancers do not do much of the real work, typically, the load on the load balancer itself should be very small. Hence, a load balancer is generally capable of fronting quite a large number of backend worker nodes. In a traditional deployment, one LB may front a cluster of homogenous worker nodes. However, a load balancer is generally capable of handling multiple clusters as shown in the figure above. The important thing to note is, the traffic has to be routed to the correct cluster, and the load has to be balanced according to the specified load balancing algorithm specified for that cluster. A cluster of homogeneous worker nodes is called a Cloud Service, in Cloud deployments. So, a load balancer which fronts multiple Cloud Services is typically called a Service-aware load balancer.



Multi-tenancy - Tenant-aware Load Balancing


If a Cloud deployment has to be able to scale to thousands, hundreds of thousands or millions of tenants, we need tenant-partitioning. This means, for a single Cloud service, there will be multiple clusters, and each of these Service clusters will handle a subset of the tenants in the system. Creating dynamic tenant clusters & tenant partitioning strategies are some of the ongoing research areas. In such a tenant-partitioned deployment, the load balancers themselves need to be tenant-aware, in order to be able to route the requests to the proper tenant clusters. This means the load balancer has to be tenant-aware as well as Service-aware, since it is the Service clusters that are partitioned according to the tenants.


Single Point of Failure?


The load balancer itself can become a single point of failure, defeating the purpose of having clustered deployments. This can be handled by having the LBs deployed in pairs in either hot-hot or hot-cold configuration. If the LBs are deployed in hot-hot configuration, we could do DNS round-robin between these LBs. In hot-cold setups, if the primary LB fails, we could do an IP remapping.


WSO2 Elastic Load Balancer


The WSO2 Load Balancer (LB) is a load balancer based on Apache Synapse & WSO2 Carbon. This is also an Elastic Load Balancer, and has been deployed on StratosLive, the Platform-as-a-Service from WSO2. It is also available for download as part of the WSO2 Stratos Cloud Middleware Platform. At present, the WSO2 Elastic Load Balancer (ELB) is only Service-aware, and can be deployed as a load balancer cluster.



Further Reading


                   









by Afkham Azeez at May 06, 2012 02:17 PM


Afkham Azeez—Is PaaS on top of IaaS the proper way to go?



Traditionally, when people talk about Cloud computing, people talk about SaaS, PaaS & IaaS as depicted in the following diagram.










So, traditionally people think about building a PaaS on top of an IaaS. Samisa recently blogged about the practical problems we have been facing with IaaS, in particular, Amazon Web Services (AWS). To be honest, working with AWS has been a big challenge since we experienced very bad performance when it comes to IO, as well as the infrastructure itself behaving in a very unpredictable manner from time to time. Every so often, we would lose the network, and we would not even be able to connect to 127.0.0.1 from our software, forcing restarts. So much for SLAs & high availability! It is also well known that virtualization leads to IO performance degradation, so if your PaaS or SaaS is IO intensive, you may see a considerable drop in performance.

Coming to think about it, running a PaaS on top of an IaaS such as AWS could be overkill. In such a setup, we have two levels of multi-tenancy; one at the IaaS layer, where the PaaS service provider is a tenant, the other level is at the PaaS layer itself. One level of multi-tenancy at the PaaS level is what is actually required. When it comes to elasticity in a PaaS, what we actually need is a new process; in the case of Java PaaS, a new JVM, but what we do in a setup such as the one shown in the above figure is, we spin up a new image instance (in the case of AWS, a new EC2 instance), and then start a process in that new instance. Spinning up a new instance can take up to 15 minutes, so by the time a new instance boots up & is able to perform some work, the need for starting up that instance may have passed due to the traffic dropping back to normal levels.






     



Perhaps, the proper model would be to run your PaaS on the infrastructure (hardware+networking+OS) directly without virtualization as shown in the above diagram, and have a few cold standby EC2 instances for Cloud bursting. This is the model we will have to go with, at least until the time where we have IaaSs that are much more stable. Another advantage is, the cost of having your own hardware will be very much less than the accumulated amount you would be paying the IaaS provider, since you will be running your PaaS 24x7.




The challenges of going for such a setup include, having to implement alternatives to many of the functionality that is already provided by the IaaS. This includes geographically distributed deployments (AWS provides this through availability zones & regions), firewall functionality (AWS provide this using security groups), public IP address assignment (AWS provides this through Elastic IPs), and so on. However, the benefits of implementing such functionality at the infrastructure level will yield huge benefits for large scale Platforms-as-a-Service such as StratosLive.


by Afkham Azeez at May 06, 2012 02:14 PM


Afkham Azeez—This blog template uses SyntaxHighlighter hosted on StratosLive












I have included SyntaxHighlighter hosted on WSO2 StratosLive under my tenant azeez.org in this blog's template as shown in the above figure. If you'd like to include syntax highlighting on your blog & want to use this hosted SyntaxHighlighter, you need to add the following in the head section of your template:



             
<link href=" http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Styles/SyntaxHighlighter.css" rel="stylesheet" type="text/css"></link>
<script src=" http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shCore.js" type="text/javascript"/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushCpp.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushCSharp.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushCss.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushJava.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushJScript.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushPhp.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushPython.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushRuby.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushSql.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushVb.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushXml.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushPerl.js' type='text/javascript'/>

<script language='javascript'>
    window.onload = function () {
        dp.SyntaxHighlighter.ClipboardSwf = '/Scripts/clipboard.swf';
        dp.SyntaxHighlighter.HighlightAll('code');
     }
</script>

Now, whenever you need to include highlighted code, you simply have to do the following;

<pre name="code" class="c-sharp">
... some code here ...
</pre>

For more information, see http://code.google.com/p/syntaxhighlighter/wiki/Usage

by Afkham Azeez at May 06, 2012 02:06 PM


Afkham Azeez—How to setup WSO2 Elastic Load Balancer





In "WSO2 Load Balancer - how it works" we discussed the internal workings of the WSO2 Elastic Load Balancer. In this article, we will discuss in detail how to configure this load balancer. Even though not necessary, understanding how the WSO2 load balancer works will enable the reader to understand the rationale behind the way certain things are configured.


1.0 Example Setup


Let us take the example of configuring the following setup shown in the diagram below. We will look at how to configure the elastic load balancer as well as the worker node clusters in this setup. By understanding this setup, you should be able to setup WSO2 ELB fronted clusters with any of the WSO2 Carbon based products or WSO2 Stratos Services.






In the above setup, we will have a LB cluster setup in primary-secondary configuration. This LB cluster will be fronting 2 clusters; App Server cluster & Data Services Server cluster.



In this exercise, we will assume the following ports.



Load Balancer

HTTP port: 80

HTTPS port: 443



App Server

HTTP port: 9762 --> proxied via port 80

HTTPS port: 9443 --> proxied via port 443



Data Services Server

HTTP port: 9762 --> proxied via port 80

HTTPS port: 9443 --> proxied via port 443




We will also assume well-known address based membership discovery (wka) to keep the article simple. The same configuration will work with multicast based membership discovery too, with the membershipScheme set to multicast in the axis2.xml file.


In the following sections, we will look at how to configure the worker node clusters & the ELB.



2.0 Configuring the App Server Cluster


You will have to change the mgt-transports.xml, carbon.xml & axis2.xml file in order to configure the App Server cluster. These files are available in the CARBON_HOME/repository/conf directory.


2.1 mgt-transports.xml


Uncomment the HTTP proxy port in this file. The HTTP proxy port has to be set to port 80. In this case, the LB is the HTTP proxy, and we are informing the App Server, that it is fronted by a proxy where the HTTP port is 80.


 
<transport name="http" class="org.wso2.carbon.server.transports.http.HttpTransport">
    <parameter name="proxyPort">80</parameter>
</transport >

Uncomment the HTTPS proxy port in this file. The HTTPS proxy port has to be set to port 443. In this case, the LB is the HTTPS proxy, and we are informing the App Server, that it is fronted by a proxy where the HTTPS port is 443.



 
 <transport name="http" class="org.wso2.carbon.server.transports.http.HttpTransport">
    <parameter name="proxyPort">443</parameter>
 </transport >





2.2 carbon.xml


In this file, we have to specify the Host of the App Server. In this example, it is appserver.stratoslive.wso2.com. This is configured by uncommenting the HostName entry in the carbon.xml file and specifying the host as shown below.
<HostName>appserver.stratoslive.wso2.com</HostName>





2.3 axis2.xml





In the axis2.xml file, you have to enable the clustering. This is used for mainly membership discovery. As explained in "WSO2 Load Balancer - how it works", the LB discovers worker nodes using the underlying Axis2 clustering mechanism.
      
<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
    <parameter name="membershipScheme">wka</parameter>
    <parameter name="localMemberHost">appserver.private.stratoslive.wso2.com</parameter> 
    <parameter name="localMemberPort">4100</parameter>
    <parameter name="domain">wso2.as.domain</parameter>
    <members>
        <member>
            <hostName>appserver.stratoslive.wso2.com</hostName>
            <port>4000</port>
        </member>
    </members>
</clustering>


Note that appserver.stratoslive.wso2.com maps to the public IP address of the load balancer. The localMemberHost can be an IP address which has private visibility. You can keep the localMemberHost entry empty, and if it is empty, one of the available interfaces of the machine the App Server node is running on will be picked. The important thing to note here is that the clustering domain has been set to wso2.as.domain.




3.0 Configuring the Data Services Server Cluster


You will have to change the mgt-transports.xml, carbon.xml & axis2.xml file in order to configure the Data Services Server cluster, and the configuration is almost the same as that of the App Server cluster.


3.1 mgt-transports.xml


The configuration is identical to what we saw in section 2.1 in the App Server cluster configuration section.


3.2 carbon.xml


In this file, we have to specify the Host of the Data Services Server. In this example, it is data.stratoslive.wso2.com. This is configured by uncommenting the HostName entry in the carbon.xml file and specifying the host as shown below.
<HostName>data.stratoslive.wso2.com</HostName>





3.3 axis2.xml



Most of the configuration is identical to what we've seen in section 2.3. The main difference will be that the clustering domain has to be set to wso2.ds.domain. The relevant configuration is shown below.
<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
    <parameter name="membershipScheme">wka</parameter>
    <parameter name="localMemberHost">data.private.stratoslive.wso2.com</parameter> 
    <parameter name="localMemberPort">4100</parameter>
    <parameter name="domain">wso2.ds.domain</parameter>
    <members>
        <member>
            <hostName>data.stratoslive.wso2.com</hostName>
            <port>4000</port>
        </member>
    </members>
</clustering>

Note that appserver.stratoslive.wso2.com maps to the public IP address of the load balancer.




4.0 Configuring the Elastic Load Balancer Cluster




axis2.xml


In this file, locate the transportReceiver entries for the HTTP & HTTPS ports and change them to port 80 & 443 respectively. The relevant XML segments are shown below.
<transportReceiver name="http" class="org.apache.synapse.transport.nhttp.HttpCoreNIOListener">
    <parameter name="port" locked="false">80</parameter>
</transportReceiver>

<transportReceiver name="https" class="org.apache.synapse.transport.nhttp.HttpCoreNIOSSLListener">
    <parameter name="port" locked="false">443</parameter>
</transportReceiver>






loadbalancer.xml


Generally, you will only need to change the CARBON_HOME/conf/loadbalancer.xml file. All other relevant files have the proper defaults, and will have to be changed only in the case of advanced configurations.


The following segment in the loadbalancer.xml file shows how we map the HTTP host to the clustering domain.

<loadBalancerConfig xmlns="http://ws.apache.org/ns/synapse">
    <services>
        <service>
            <hosts>
                <host>appserver.stratoslive.wso2.com</host>
            </hosts>
            <domain>wso2.as.domain</domain>
        </service>
        <service>
            <hosts>
                <host>data.stratoslive.wso2.com</host>
            </hosts>
            <domain>wso2.ds.domain</domain>
        </service>
    <services>
</loadBalancerConfig>





4.1 Disabling Autoscaling


At the time of writing, the WSO2 Elastic Load Balancer only supports autoscaling on Amazon EC2 & compatible IaaSs since it is based on the EC2 API. So, you will see error messages if you try to run this setup on a local network, for example. So, in such cases, you will have to disable autoscaling. You have to do the following to disable autoscaling;



Delete $CARBON_HOME/repository/deployment/server/synapse-configs/tasks/autoscaler.xml
Remove the autoscaleIn & autoscaleOut mediators from $CARBON_HOME/repository/deployment/server/synapse-configs/sequences/main.xml

5.0 Testing the Setup


Now we have completed setting up the system, so it is just a matter of starting up the nodes. If you have autoscaling setup, you will simply have to start only the Elastic Load Balancer. This LB will take care of starting up all other nodes in the system. If everything is properly setup, you should see messages on the LB log file saying members from wso2.as.domain & wso2.ds.domain joined the cluster.



To make sure that everything is working properly, in your client machine, map the appserver.stratoslive.wso2.com & data.stratoslive.wso2.com names to the relevant IP address of the primary Elastic load balancer & go to the following URLs using your Web browser.



http://data.stratoslive.wso2.com
http://appserver.stratoslive.wso2.com




If you have properly setup your cluster, you should see the login screens of the relevant WSO2 product.




by Afkham Azeez at May 06, 2012 02:00 PM


Afkham Azeez—My Top 10 Third Party IntelliJ IDEA Plugins


















Hungry backspace
FindBugs-IDEA
CopyFQNPlugin
GrepCode
Copy As HTML
SOAP UI
TCP Monitor
BashSupport
Generate toString
Atlassian Connector



by Afkham Azeez at May 06, 2012 01:52 PM


Afkham Azeez—Lazy Loading Deployment Artifacts in a PaaS Deployment




1.0 Introduction

One of the fundamental goals of Cloud computing is optimal usage of limited resources. This involves maximizing ROI for all parties concerned, and making sure that all parties win. Multi-tenancy is a core attribute of a Cloud deployment. We have to ensure that all available resources can be optimally utilized by many tenants as possible. Lazy loading plays a major role in resource usage optimization.




In this article, we will take a look at how deployment artifacts (webapps, service artifacts etc) can be optimally loaded in a Platform-as-a-Service (PaaS) implementation. We will describe how we have implemented lazy loading in the WSO2 Stratos Cloud Middleware Platform & the WSO2 StratosLive PaaS.




2.0 Lazy Loading

Lazy loading is a widely used & well-known concept in the field of computing. There are four different variants of the lazy loading pattern, as documented by Martin Fowler. The differences between these variants are subtle.




1. Lazy Initialization

A null field indicates that there is no data. When the value is requested, a null check is performed to see if the actual data needs to be loaded.




2. Virtual Proxy

The virtual proxy implements the same interface as the real object, and when called for the very first time, it loads the real object & delegates to that object.




3. Value Holder

A value holder is an object with a getValue method, which the clients will invoke in order to obtain a reference to the real object. Note that the method may not necessarily be named getValue.




4. Ghost

The real object without any data. The data is loaded as and when required.




In the next sections of the article, we will explain how we used the Value Holder & Ghost variants of the lazy loading pattern in the Stratos Cloud Middleware Platform & the StratosLive PaaS.







3.0 Lazy Loading Tenant AxisConfiguration

The Stratos deployment model revolves around the Axis2 deployment model. We have found the Axis2 deployment model to be a very flexible & powerful one, hence we decided to go ahead with the Axis2 deployment model for deploying artifacts in the WSO2 Carbon middleware platform & WSO2 Stratos Cloud middleware platform. The Axis2 deployment model totally isolates artifact deployment & classloader isolation, hence is a very good model to adopt for a multi-tenanted Cloud deployment.




When the WSO2 Stratos processes start up, we do not load all the tenants into memory. Obviously, it will not scale if we were to do so. Tenants are loaded only when they are needed, and if a tenant is not being used for a certain time period, we unload such tenants. Tenants will be loaded in the following scenarios;




1. A request is received for an artifact deployed by the tenant. 

These can be requests to webapps, Web services, proxy services, tenant UI components and so on.




2. A tenant's management console view is loaded. 

Tenant's can manage services in their view of the PaaS using the management console. When tenants want to perform such management activities, we load the tenant.




3. A periodic task scheduled for the tenant gets activated

In a PaaS, tenants should be able to schedule tasks that run periodically. In such cases, when the task starts to run, if the tenant has not been loaded, we will load the tenant.




In WSO2 Stratos, loading a tenant mainly translates to creation of an Axis2 Configuration & ConfigurationContext in programming terms. When an AxisConfiguration is created for a tenant, a series of OSGi service calls results in the relevant permission scheme & other metadata being loaded. These are lightweight service calls that do not impose high overhead.




We have used the ValueHolder variant of lazy loading in this instance. The relevant method call is;




TenantAxisUtils.getTenantConfigurationContext(String tenantDomain)




The getTenantConfigurationContext method is the equivalent of the getValue method in the ValueHolder. This method will create & return a new Axis2 ConfigurationContext for that tenant domain, if it does not already exist, and will return the already created ConfigurationContext if the tenant has already been loaded.




One shortcoming of loading the entire AxisConfiguration is that all artifacts belonging to that particular tenant will also be loaded. Hence, the time taken for the execution of this method will increase with the number of artifacts deployed by the tenant. This can lead to unfavorable consequences such as the very first request received for a tenant always timing out. Such requests are targeted at a single artifact, but loading the AxisConfiguration means the entire Axis2 repo gets loaded, hence we will end up loading all artifacts, leading to unnecessary memory usage & processing time. In section 4.0, we will describe how we can overcome this issue by using the Ghost variant of the lazy loading pattern.




4.0 Lazy Loading Tenant Deployment Artifacts

As described in section 3.0 above, loading an entire tenant AxisConfiguration can lead to increasing first-request response times since all artifacts have to be loaded. We can overcome this by loading the required artifact into a tenant's AxisConfiguration on demand.




We use the Ghost variant of the lazy loading pattern to accomplish this. Let's take a look at how normal artifact deployment works.








Figure 1: Deployment without Ghost Deployer & Ghost Dispatcher




As shown in the above diagram, a set of Axis2 deployers are registered for each artifact type. The DeploymentEngine maintains a collection of such deployers, and then when an AxisConfiguration is created, it will call all the deployers to load all the artifacts found in the Axis2 repository. The deployers will load all the artifacts into memory. Needless to say, this will take a long time if there are many artifacts to be loaded. Next let us look into the concept of Ghost artifact deployment, to see how we reduce this initial loading time.







Figure 2: Ghost Deployment with Ghost Deployer & Ghost Dispatcher





As shown in the above diagram, we change the deployment slightly & register GhostDeployers for all artifact types. The GhostDeployer instance will maintain references to the actual Axis2 deployers. Let us look at the sequence of events.





1. When a new artifact is deployed, either by uploading using the Management console UI, or copying the artifact to the Axis2 repo, the GhostDeployer will call the real deployer, which will create the real Axis2 Service




2. The GhostDeployer will then pick up that Axis2 service, and create a simple metadata file which has some basic information needed to create a Ghost Axis2Service. This will mainly contain the service name & operations; information needed for dispatching requests.




3. On subsequent deployments of the service which was first created in step 2, the GhostDeployer will pick it up from the metadata file created in step 2 above.




4. Next the GhostDeployer will create a Ghost Axis2 Service which will contain a special parameter (Ghost Parameter) which will identify the service as a Ghost service, and register that service into the AxisConfig




5. Next, say a request comes in for the said service, which at the moment is deployed in Ghost form.




6. There is a special Axis2 dispatching handler called the GhostDispatcher, which is registered after all standard Axis2 dispatchers. It will check the Ghost Parameter added to the service in step 4 above to check whether the Actual service deployment needs to take place, if the service in concern is a Ghost service.




7. If the service is determined to be a Ghost service in step 6 above, the GhostDispatcher will call the GhostDeployer to carry out the actual service deployment. At that point, the GhostDeployer will call the actual deployer, which will load the rest of the metadata & information required by the actual service. This will load all metadata from the registry, load service policies & so on.




In addition, if certain services are not used for some time, they will be undeployed & reloaded in Ghost form. This will ensure that rarely used services, or services that don't get called too often, do not unnecessarily use up resources.




5.0 Performance Comparison




We carried out performance comparisons & plotted the graph of "Initial Response Time" vs. "Number of Artifacts". The initial response time is the response time seen by a client when a tenant has not been loaded. The number of artifacts is the number of deployment artifacts deployed under that tenant.




We sent requests targeted at a particular operation in a particular service. We recorded the initial response times while linearly increasing the number of deployment artifacts up to 300 in that tenant. The resulting graph is shown below.





Figure 3: Performance comparison - with & without Ghost Deployment


As can be seen, without Ghost Deployment, the initial response time linearly increases. Eventually, clients sending the very first requests will start timing out. With Ghost Deployment, we ensure that the first response time is linear, hence this will improve the client experience on first request, and will also ensure that the first requests don't timeout.


Ghost Deployment is one of the major enhancements we introduced in the recent StratosLive upgrades to enable our tenants to experience better performance.



6.0 Conclusion

Lazy loading is a core requirement in Cloud deployments. In this article we looked at some of the core concepts behind lazy loading. We saw how Ghost Deployment yields drastic performance improvements in a multi-tenanted PaaS environment such as StratosLive. Future enhancements in this area in Stratos will include lazy loading of webapps using a GhostWebappDeployer Tomcat valve.

by Afkham Azeez at May 06, 2012 01:20 PM


Afkham Azeez—WSO2 ESB - Service Chaining



This is a follow up to the article titled "WSO2 ESB by Example - Service Chaining" by Supun Kamburugamuva which describes in detail how the following service chaining scenario can be implemented using WSO2 ESB.









Since that article was written, there has been an enhancement to the WSO2 ESB send mediator where the send mediator can direct the response to a named sequence. This simplifies the configuration quite a bit since it is no longer necessary to handle all responses in the outSequence, and decide what to do next depending on properties.



The new proxy service configuration looks like the following:



      <proxy name="CreditProxy" transports="https http" startOnLoad="true" trace="disable">
        <description/>
        <target>
            <inSequence>
                <log>
                    <property name="sequence" value="inSequence - request for CreditProxy"/>
                </log>
                <property xmlns:sam="http://samples.esb.wso2.org" name="ORG_ID" expression="//sam:credit/sam:id"/>
                <property xmlns:sam="http://samples.esb.wso2.org" name="ORG_AMOUNT" expression="//sam:credit/sam:amount"/>
                <enrich>
                    <source type="inline" clone="true">
                        <sam:get xmlns:sam="http://samples.esb.wso2.org">
                            <sam:id>?</sam:id>
                        </sam:get>
                    </source>
                    <target type="body"/>
                </enrich>
                <enrich>
                    <source type="property" clone="true" property="ORG_ID"/>
                    <target xmlns:sam="http://samples.esb.wso2.org" xpath="//sam:get/sam:id"/>
                </enrich>
                <log>
                    <property name="sequence" value="inSequence - request for PersonInfoService"/>
                </log>
                <send receive="personInfoSeq">
                    <endpoint key="PersonInfoEpr"/>
                </send>
            </inSequence>
            <outSequence>
                <drop/>
            </outSequence>
        </target>
        <publishWSDL uri="file:resources/CreditProxy.wsdl"/>
    </proxy>




The two named sequences are shown below:



    <sequence name="personInfoSeq">
        <xslt key="xslt">
            <property name="amount" expression="get-property('ORG_AMOUNT')"/>
        </xslt>
        <send receive="creditSeq">
            <endpoint key="CreditEpr"/>
        </send>
    </sequence>




    <sequence name="creditSeq">
        <log level="full"/>
        <send/>
    </sequence>




The send mediator in the inSequence of the CreditProxy service defines the receiving sequence of the response as personInfoSeq. So the response will be forwarded to the personInfoSeq instead of the outSequence. In the personInfoSeq, we have a send mediator which defines the receiving sequence as creditSeq, which will simply send the response out to the client.



As you will notice, this is a major improvement when it comes to writing a service chaining configuration over the previous method of using a STATE property combined with switch & case mediators.

by Afkham Azeez at May 06, 2012 01:10 PM


Afkham Azeez—The birth of Stratos multi-tenancy







On Nov 14th, 2009, I was all by myself & had a LONG & BORING 12hr transit at the Hong Kong airport! I was returning to Sri Lanka after a trip to the US, where I was a speaker at ApacheCon NA, 2009.














Since I had nothing better to do, I started looking at how multi-tenant service deployment & dispatching can be made to work in WSO2 WSAS (since then, this product has been renamed to WSO2 AppServer). I tweeted about what I was planning to do.














I started implementing this, and got very excited when the first request got dispatched to a tenant. So, again I tweeted about this (now historic) event. 













This was the birth of the WSO2 Stratos Cloud middleware platform! The code I wrote during this transit formed the basis for multi-tenancy in Stratos.




During the next month, I got some of the other core stuff working, and like any good Tweep, tweeted about the progress.











My tweets had gotten Sanjiva excited, prompting him to send me the following mail.











The very first bits of the multi-tenancy code was developed in a scratch section of the WSO2 Subversion repo before it became stable enough to be committed to the trunk.



So, it's been 2 years since the first message got dispatched to a tenant at the HKG airport. Time well spent during a long transit. Over the past few years, this code was improved by many people & Stratos  has grown from strength to strength to become the only complete open source Cloud middeware platform. StratoLive, the PaaS based on Stratos, went onto to become the first & only complete PaaS.



P.S. I used SnapBird to search through my old tweets.

by Afkham Azeez at May 06, 2012 01:09 PM


Afkham Azeez—6 years at WSO2!



3 years ago, I wrote a blog post titled "3 years at WSO2". So, it's only befitting that I write a similar post at the end of 3 more years. These 3 years passed by very quickly. Time does fly when you are enjoying yourself.



Over the past 3 years, the company has grown from strength to strength while continuing the maintain its culture & values, and emerged as a leader in the middleware space. We've built a comprehensive middleware stack, as well as built the first complete Cloud middleware platform.



Even after 6 long years, I'm still having a lot of fun & always learning something new all the time, which keeps me excited & motivated.







by Afkham Azeez at May 06, 2012 12:43 PM


Kathiravelu Pradeeban—"Reading Level" ~ Google gets more interesting!
"Verbatim" and "Reading Level" are two new cool search tools from Google. Verbatim lets us search the exact term as it is, eliminating the spelling corrections, and all the smart features of Google.






Reading Level

"Reading
 Level" categorizes the search results according to the level of 
expertise needed in reading the page. Simple and easy-to-comprehend 
pages are categorized as "Basic", where the advanced and high-standard 
pages are categorized "Advanced". "Intermediate" stays in between. 
Highly technical or articles with complex language structures are often 
categorized as "Advanced".




I searched "Kathiravelu Pradeeban" using Reading Level, and found the below.

Basic - 10%

Intermediate  - 80%

Advanced - 11%




Similarly, searched a few projects that I am interested in.

AbiWord 

Basic - 14%

Intermediate  - 83%

Advanced - 3%




OGSA-DAI

Basic < 1%

Intermediate 11%

Advanced 89%




It
 is pretty reasonable to have the highly complex and technically 
advanced OGSA-DAI to have more pages classified as "Advanced" and only a
 tiny bit classified as "Basic". However, AbiWord, the word processor 
has more contennt classified as "Intermediate". This again is reasonable
 for an end-reason software product. Google Reading Level is surely an 
interesting feature. 

by Kathiravelu Pradeeban at May 06, 2012 07:20 AM

May 05, 2012


Kathiravelu Pradeeban—[IJCISIM] Horizontal Format Data Mining with Extended Bitmaps

We published our data mining algorithm to the International Journal of  Computer Information Systems and Industrial Management Applications ISSN 2150-7988. It is available online with the Volume 4 - 2012.





Feel free to download it.
Horizontal Format Data Mining with Extended Bitmaps

Buddhika De Alwis, Supun Malinga, Kathiravelu Pradeeban, Denis Weerasiri, Shehan Perera

pp. 514-521 Full        Text PDF






Abstract: Analysing the data warehouses to foresee the patterns of the transactions often needs high computational power and memory space due to the huge set of past history of the data transactions. With the fragmented data along with the current trend of distributed systems, most of the fundamental algorithms that are initially proposed to find the association among the itemsets in the data warehouses are inefficient either in throughput or the utilization of the resources.

Apriori algorithm is a mostly learned and implemented algorithm that mines the data warehouses to find the associations. However, Apriori is generally not an optimized algorithm. More variations, improvements, and alternatives have been suggested to overcome the inefficiency of Apriori algorithm, either as a whole or to specific sets of data. In any case, a fraction of improvement in the algorithm often improves the mining considerably. Frequent item set mining with vertical data format has been proposed as an improvement over the basic Apriori algorithm, which mines the data sets of vertical form, opposed to the typical horizontal format data as in case of Apriori.

In this paper we are proposing an algorithm as an alternative to Apriori algorithm, which will use bitmap indices in conjunction with a horizontal format data set converted to a vertical format data structure to mine frequent itemsets leveraging efficiencies of bitmap based operations and vertical format data orientation.



Keywords: Data mining, Association Rule, Apriori, Vertical format mining, Bitmap Indices, Data Analysis, Data Warehousing. 


by Kathiravelu Pradeeban at May 05, 2012 09:10 AM


Chamara Ariyarathne—OpenLDAP Clustering Guide
This is a complete guide for OpenLDAP installing and clustering in Mirror Mode.

First you need to have installed BerkeleyDB as the data-store for OpenLDAP. You can Download BerkeleyDB from 
http://www.oracle.com/technetwork/products/berkeleydb/downloads/index.html

Also make sure that you need g++ and all the essential dependencies installed in your machine. If you are  using ubuntu

$sudo apt-get install build-essential will install all the dependencies. 

Here I have used BDB 4.8.30 version.

1. Create a directory for BDB installation
mkdir /home/chamara/OpenLDAP/BerkeleyDB

2. Unzip the BerkeleyDB distribution
tar -xvf db-4.8.30.tar.gz

3. Go to the directory 
/db-4.8.30/build_unix 

4. Run the following command
/build_unix$ ../dist/configure --prefix=/home/chamara/OpenLDAP/BerkeleyDB

--prefix will set the final BDB installation path. There are lots of parameters which can be set at the installation. For a complete reference please refer the BDB documentation.

5. Finally you will get the following output in the end of configure
configure: creating ./config.status
config.status: creating Makefile
config.status: creating db_cxx.h
config.status: creating db_int.h
config.status: creating clib_port.h
config.status: creating include.tcl
config.status: creating db.h
config.status: creating db_config.h
config.status: executing libtool commands

Also if you see the current directory there will  be a make file created

-rw-r--r--  1 chamara chamara  81K 2012-05-05 05:37 Makefile

So that now you can work on the installation by continuing the following procedure

$make
$make install

Now if you check the BerkeleyDB directory

/BerkeleyDB$ ls -lah
total 24K
drwxr-xr-x  6 chamara chamara 4.0K 2012-05-05 05:51 .
drwxr-xr-x  5 chamara chamara 4.0K 2012-05-05 05:36 ..
drwxr-xr-x  2 chamara chamara 4.0K 2012-05-05 05:51 bin
drwxr-xr-x 13 chamara chamara 4.0K 2012-05-05 05:51 docs
drwxr-xr-x  2 chamara chamara 4.0K 2012-05-05 05:51 include
drwxr-xr-x  2 chamara chamara 4.0K 2012-05-05 05:51 lib

6. Now you have to set the following parameters for OpenLDAP to find where the BerkeleyDB is installed

$CPPFLAGS="-I/home/chamara/OpenLDAP/BerkeleyDB/include" 
$export CPPFLAGS 
$LDFLAGS="-L/usr/local/lib -L/home/chamara/OpenLDAP/BerkeleyDB/lib -R/home/chamara/OpenLDAP/BerkeleyDB/lib" 
$export LDFLAGS 
$LD_LIBRARY_PATH="/home/chamara/OpenLDAP/BerkeleyDB/lib" 
$export LD_LIBRARY_PATH

* Now BerkeleyDB is installed properly

7. Now you need to have a OpenLDAP distribution. You can download from
http://www.openldap.org/software/download/

I have used  openldap-stable-20100719.tgz
Unzip the distribution
$ tar -xvf openldap-stable-20100719.tgz

Now go to the OpenLDAP distribution

$ cd openldap-2.4.23/
$ ls -alh
total 1.5M
drwxr-xr-x 10 chamara chamara 4.0K 2010-06-30 05:23 .
drwxr-xr-x  5 chamara chamara 4.0K 2012-05-05 05:36 ..
-rw-r--r--  1 chamara chamara 244K 2005-10-30 03:37 aclocal.m4
-rw-r--r--  1 chamara chamara 3.8K 2010-04-14 01:52 ANNOUNCEMENT
drwxr-xr-x  2 chamara chamara 4.0K 2012-05-05 05:36 build
-rw-r--r--  1 chamara chamara  42K 2010-06-29 20:53 CHANGES
drwxr-xr-x  3 chamara chamara 4.0K 2012-05-05 05:36 clients
-rwxr-xr-x  1 chamara chamara 1.1M 2010-04-20 00:52 configure
-rw-r--r--  1 chamara chamara  92K 2010-04-19 22:23 configure.in
drwxr-xr-x  7 chamara chamara 4.0K 2012-05-05 05:36 contrib
-rw-r--r--  1 chamara chamara 2.3K 2010-04-14 01:52 COPYRIGHT
drwxr-xr-x  8 chamara chamara 4.0K 2012-05-05 05:36 doc
drwxr-xr-x  3 chamara chamara 4.0K 2012-05-05 05:36 include
-rw-r--r--  1 chamara chamara 4.4K 2010-04-14 01:52 INSTALL
drwxr-xr-x  8 chamara chamara 4.0K 2012-05-05 05:36 libraries
-rw-r--r--  1 chamara chamara 2.2K 2003-11-25 00:42 LICENSE
-rw-r--r--  1 chamara chamara 1.1K 2010-04-14 01:52 Makefile.in
-rw-r--r--  1 chamara chamara 3.5K 2010-04-14 01:52 README
drwxr-xr-x  3 chamara chamara 4.0K 2012-05-05 05:36 servers
drwxr-xr-x  5 chamara chamara 4.0K 2010-06-30 05:23 tests

8. Run the following command
$ ./configure --prefix=/home/chamara/OpenLDAP/OpenLDAP

again, as in BDB installation --prefix will set the final OpenLDAP installation path

9. Now you will see a MakeFile  is created 
-rw-r--r--  1 chamara chamara 9.3K 2012-05-05 06:07 Makefile

So that, going through the following order you will have installed OpenLDAP

$ make depend
$ make
$ make test
$ make install

10. Now we are done with the OpenLDAP installation. If you check the destination directory

/OpenLDAP$ ls -lah
total 40K
drwxr-xr-x 10 chamara chamara 4.0K 2012-05-05 06:35 .
drwxr-xr-x  6 chamara chamara 4.0K 2012-05-05 06:06 ..
drwxr-xr-x  2 chamara chamara 4.0K 2012-05-05 06:35 bin
drwxr-xr-x  3 chamara chamara 4.0K 2012-05-05 06:35 etc
drwxr-xr-x  2 chamara chamara 4.0K 2012-05-05 06:35 include
drwxr-xr-x  2 chamara chamara 4.0K 2012-05-05 06:35 lib
drwxr-xr-x  2 chamara chamara 4.0K 2012-05-05 06:35 libexec
drwxr-xr-x  2 chamara chamara 4.0K 2012-05-05 06:35 sbin
drwxr-xr-x  3 chamara chamara 4.0K 2012-05-05 06:35 share
drwxr-xr-x  4 chamara chamara 4.0K 2012-05-05 06:35 var

Now we have to configure OpenLDAP installation. I will refer this directory as $OpenLDAP_HOME

11. change the directory to $OpenLDAP_HOME

12. Append the following in the etc/openldap/ldap.conf

BASE dc=test,dc=com
URI ldap://172.16.246.1:1389

You have to mention a valid ipaddress

12. Create DB_CONFIG file

 $ cp etc/openldap/DB_CONFIG.example etc/openldap/DB_CONFIG

13. Now configure the etc/openldap/slapd.conf

Append following;
include         /home/chamara/OpenLDAP/OpenLDAP/etc/openldap/schema/cosine.schema
include         /home/chamara/OpenLDAP/OpenLDAP/etc/openldap/schema/nis.schema
include         /home/chamara/OpenLDAP/OpenLDAP/etc/openldap/schema/inetorgperson.schema

After;
include         /home/chamara/OpenLDAP/OpenLDAP/etc/openldap/schema/core.schema

Find;
BDB database definitions

Change them into;
suffix          "dc=test,dc=com"
rootdn          "cn=admin,dc=test,dc=com"

rootpw          admin123

Add followings for the Mirror Mode Replication;

index   objectClass     eq
index entryCSN,entryUUID eq

syncrepl rid=002
        provider=ldap://{$ip-address of the other OpenLDAP instance$}:1389/
        type=refreshAndPersist
        retry="60 30 300 +"
        searchbase="dc=test,dc=com"
        bindmethod=simple
        binddn="cn=admin,dc=test,dc=com"
        credentials=admin123
mirrormode TRUE

overlay syncprov
syncprov-checkpoint 100 10
syncprov-reloadhint true
syncprov-nopresent true
syncprov-sessionlog 100




For my machine;  {$ip-address of the other OpenLDAP instance$} is
172.16.246.128

14. Now the configuration of OpenLDAP node 1 is done. Follow the same procedure for the OpenLDAP node2, and you will only have to change the ip-addresses in the ldap.conf and slapd.conf

15. Start OpenLDAP using following commands
$ ./libexec/slapd -h ldap://172.16.246.1:1389
or
$ ./libexec/slapd -h ldap://172.16.246.1:1389 -d3 (debug mode)

16. Create following files to add the default users to OpenLDAP store

$ vi build_root_ou.ldif
INSERT;

dn: dc=test,dc=com
objectClass: dcObject
objectClass: organizationalUnit
dc: test
ou: testou

$ vi add_user_ou.ldif
INSERT;

dn: ou=user,dc=test,dc=com
objectClass: organizationalUnit
ou: users

$ vi add_groups_ou.ldif
INSERT;

dn: ou=Groups,dc=test,dc=com
objectClass: organizationalUnit
ou: Groups

$ vi add_user_uid.ldif
INSERT;

dn: uid=admin,ou=user,dc=test,dc=com
cn: Admin
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: top
cn: WSO2
sn: Open Source Middleware
uid: admin
userPassword:  {SSHA}A1toNdJpoocuptlnEYkKWZa45oxag4GG

Use;
$OpenLDAP_HOME/sbin$ ./slappasswd
to encrypt the password and get the SSHA value
/sbin$ ./slappasswd 
New password: 
Re-enter new password: 
{SSHA}A1toNdJpoocuptlnEYkKWZa45oxag4GG
I used 'admin123' as password

17. Now use the following command to add the .ldif files to the ldap store
(Now I'm in the $OpenLDAP_HOME/etc/openldap dir. where all the .ldif files saved in) 

$ ../../bin/ldapadd -D "cn=admin,dc=test,dc=com" -W -x -f build_root_ou.ldif

Enter LDAP Password: 
adding new entry "dc=test,dc=com"

Follow the procedure;
$ ../../bin/ldapadd -D "cn=admin,dc=test,dc=com" -W -x -f add_user_ou.ldif
$ ../../bin/ldapadd -D "cn=admin,dc=test,dc=com" -W -x -f add_groups_ou.ldif
$ ../../bin/ldapadd -D "cn=admin,dc=test,dc=com" -W -x -f add_user_uid.ldif

18. Everything is done :)

If you can use ApacheDirectoryStudio and connect to the OpenLDAP ldap store you will see the ldap tree we created.

Network Parameters; 





 Authentication;


Now click on Finish button and you will be connected to the ldap store. You can browse and add users from there. 



by Chamara Anuradha at May 05, 2012 02:16 AM

May 04, 2012


Prabath Siriwardena—Running two OpenLDAP instances in the same machine under MAC OS X
This blog post explains how to run two OpenLDAP instances in the same machine under MAC OS X.

1. Setup the first instance of OpenLDAP as explained in my previous blog post.

2. Execute the following commands in the same order.

$ sudo cp -r /private/etc/openldap /private/etc/openldap.node2

$ sudo cp -r /var/db/openldap /private/etc/openldap.node2

$ sudo rm -r  /var/db/openldap.node2/openldap-data/*db.*

$ sudo rm -r  /var/db/openldap.node2/openldap-data/*.bdb

$ sudo rm -r  /var/db/openldap.node2/openldap-data/log*.*

$ sudo rm -r  /var/db/openldap.node2/openldap-data/alock

$ sudo cp  -r  /var/db/openldap.node2/openldap-data/DB_CONFIG.example  /var/db/openldap.node2/openldap-data/DB_CONFIG

3. Open up /private/etc/openldap.node2/ldap.conf and change the port, say to 12389

4. Open up /private/etc/openldap.node2/slapd.conf and change all the references from /private/etc/openldap to /private/etc/openldap.node2

5.  Open up /private/etc/openldap.node2/slapd.conf and change all the  references from /var/db/openldap to /var/db/openldap.node2

6. Start the first OpenLDAP server running on the default port.

$ sudo /usr/libexec/slapd -d3

6.  Start the second OpenLDAP from the following command.

 $ sudo /usr/libexec/slapd -f /private/etc/openldap.node2/slapd.conf -h ldap://localhost:12389  -d3
by Prabath at May 04, 2012 07:37 AM


Prabath Siriwardena—Setting up OpenLDAP under MAC OS X
This blog post explains how to setup OpenLDAP under Mac OS X and I have tried this out successfully under OS X Lion.

First we need to install the correct Xcode version corresponding to the OS X and then the latest MacPorts. Once this is done installing OpenLDAP via MacPorts is quite simple.

% sudo port -d selfupdate

%  sudo port install openldap

The above will install OpenLDAP with Berkly DB back-end.

You will find the OpenLDAP configuration files at /private/etc/openldap

We need to worry about two configuration files here - slapd.conf and ldap.conf. You will find these two config files as slapd.conf.default and ldap.conf.default, in that case rename those to be slapd.conf and ldap.conf. Also make sure you copy the /private/var/db/openldap/openldap-data/DB_CONFIG.example to  /private/var/db/openldap/openldap-data/DB_CONFIG.

First let's open up ldap.conf. There you need set the BASE for LDAP tree - and also the URI for the LDAP server. That's all - change those settings and save the file. 

BASE dc=wso2,dc=com
#URI ldap://ldap.example.com ldap://ldap-master.example.com:666
URI ldap://192.168.1.83:389

#SIZELIMIT 12
#TIMELIMIT 15
#DEREF  never
TLS_REQCERT demand 

Next we need to modify the slapd.con file. This is one of the main LDAP configuration files.

Please make sure all related schema includes are there.. un-commented.. 

Then you need to set suffix, rootdn and rootpw.

suffix needs to be the same as what you defined for BASE in ldap.conf.

rootdn is the DN of the OpenLDAP root user. Here I have it as cn=admin,dc=wso2,dc=com.

Then the rootpw...

This is bit tricky and most people get this wrong.

If you just put any clear text value to rootpw - then when you try do an ldapsearch and try to authenticate, it will fail with the following error.

ldap_bind: Invalid credentials (49)

The reason is, the default distribution which comes with MacPorts, is built with clear text passwords being disabled. So you need to  generate the password in SHA first and then put it in to the slapd.conf. To generate the SHA password you can use the following command.

% slappasswd -s your-password

Also make sure that following two lines are un-commented...

modulepath /usr/libexec/openldap
moduleload back_bdb.la

Following is the complete slapd.conf file.

#
# See slapd.conf(5) for details on configuration options.
# This file should NOT be world readable.
#
include  /private/etc/openldap/schema/core.schema
include         /private/etc/openldap/schema/cosine.schema
include         /private/etc/openldap/schema/nis.schema
include         /private/etc/openldap/schema/inetorgperson.schema

# Define global ACLs to disable default read access.

# Do not enable referrals until AFTER you have a working directory
# service AND an understanding of referrals.
#referral ldap://root.openldap.org

pidfile  /private/var/db/openldap/run/slapd.pid
argsfile /private/var/db/openldap/run/slapd.args

# Load dynamic backend modules:
modulepath /usr/libexec/openldap
moduleload back_bdb.la
# moduleload back_hdb.la
# moduleload back_ldap.la

# Sample security restrictions
# Require integrity protection (prevent hijacking)
# Require 112-bit (3DES or better) encryption for updates
# Require 63-bit encryption for simple bind
# security ssf=1 update_ssf=112 simple_bind=64

# Sample access control policy:
# Root DSE: allow anyone to read it
# Subschema (sub)entry DSE: allow anyone to read it
# Other DSEs:
#  Allow self write access
#  Allow authenticated users read access
#  Allow anonymous users to authenticate
# Directives needed to implement policy:
# access to dn.base="" by * read
# access to dn.base="cn=Subschema" by * read
# access to *
# by self write
# by users read
# by anonymous auth
#
# if no access controls are present, the default policy
# allows anyone and everyone to read anything but restricts
# updates to rootdn.  (e.g., "access to * by * read")
#
# rootdn can always read and write EVERYTHING!

#######################################################################
# BDB database definitions
#######################################################################

database bdb
suffix  "dc=wso2,dc=com"
rootdn  "cn=admin,dc=wso2,dc=com"
# Cleartext passwords, especially for the rootdn, should
# be avoid.  See slappasswd(8) and slapd.conf(5) for details.
# Use of strong authentication encouraged.
rootpw  {SSHA}BqYQBS48EZlLu4XYJxEXaOlRdseW2D4Y
# The database directory MUST exist prior to running slapd AND 
# should only be accessible by the slapd and slap tools.
# Mode 700 recommended.
directory /private/var/db/openldap/openldap-data
# Indices to maintain
index objectClass eq

Once the above is done - we can start our OpenLDAP server...

% sudo /usr/libexec/slapd -d3

Now, we need to build our LDAP tree structure...

Save the following in to a file called root-ou.ldif.

dn:dc=wso2,dc=com
objectClass:dcObject
objectClass:organizationalUnit
dc:wso2
ou:WSO2
Now run the following command...

% ldapadd -D "cn=admin,dc=wso2,dc=com" -W -x -f root-ou.ldif

"cn=admin,dc=wso2,dc=com" is the value of rootdn that we setup in slapd.conf. When prompted for password, you can give the rootpw.

Now, let's add a OU called people under this.

Once again, save the following to a file called people-ou.ldiff. 
dn: ou=people,dc=wso2,dc=com
objectClass: organizationalUnit
ou: people

Now run the following command...

% ldapadd -D "cn=admin,dc=wso2,dc=com" -W -x -f people-ou.ldif

If your OpenLDAP instance is running on a different port than the default one - we need to use the following command instead of the above.

% ldapadd -D "cn=admin,dc=wso2,dc=com" -H ldap://localhost:389 -W -x -f people-ou.ldif

This will create a OU structure as shown in the image below.. Basically you can connect Apache Directory Studio to your running OpenLDAP instance to view it.

















Everything should be fine by now...

OpenLDAP comes with set of default schema files, which you can find inside  /private/etc/openldap/schema. If you want to have your own schema loaded in to OpenLDAP, what you have to do is, write your schema file and copy it to  /private/etc/openldap/schema and edit the slapd.conf to add an include pointing to your schema file. Then you need to restart the OpenLDAP server.

To stop the OpenLDAP instance you can use the following command...

% sudo kill  $(cat /private/var/db/openldap/run/slapd.pid) 

/private/var/db/openldap/run/slapd.pid is the place where the process id of the OpenLDAP process being stored - and this location can be configured in slapd.conf.
by Prabath at May 04, 2012 07:02 AM


Srinath Perera—Introduction to Open Source, Apache and Apache Way
Introduction to Open Source, Apache and Apache Way 
                       

This is the slide deck for introduction to Open Source and Apache Way talk I did at Apache Bar Camp 2012 at Engineering Faculty, University of Peradeniya. More info at http://readme.lk/apache-meetup-kandy/

by Srinath Perera at May 04, 2012 06:02 AM


Prabath Abeysekera—Retrieve the classes implementing a particular interface programmatically in Java
I recently had a requirement to retrieve all the classes that implement a particular service interface programmatically in Java. While searching for some way to get my requirement fulfilled I came across the class ServceLoader exposed by java.util.* package as a part of Java 1.6 APIs.  

Shown below is a sample scenario where you can use the aforementioned class effectively to retrieve the service providers (typically implementer classes of an interface or an abstract class) list that implement a particular service (typically a well known interface or abstract classs).

Assume I want to register the set of implementer classes of the java.sql.Driver interface that exists in my current thread's classloader explicitely. You can accomplish that task in the following manner.

NOTE:
If a particular service provider implements a service interface then the jar containing those implementer classes carries the provider-configuration file in the resource directory similar to the following format.

Eg: META-INF/services/java.sql.Driver 


 

by Prabath Abeysekara at May 04, 2012 05:28 AM


Sagara Gunathunga—Release of Axis2 1.6.2 and  Sandesha2 1.6.2 , Rampart 1.6.2

The Apache Axis2 team is pleased to announce the general availability of the following releases:






Axis2 1.6.2
Sandesha2 1.6.2
Rampart 1.6.2





Apache
 Axis2 is a complete re-design and re-write of the widely used Apache 
Axis engine and is a more efficient, more scalable, more

modular and more XML-oriented Web services framework. It is carefully 
designed to support the easy addition of plug-in "modules" that extend

its functionality for features such as security and reliability.



Apache Rampart is an Axis2 module that implements the specifications in the WS-Security stack.



Apache Sandesha2 provides WS-ReliableMessaging support for Axis2.



Axis2 1.6.2 is a maintenance release that contains more than 45 fixes.



Rampart 1.6.2 is a maintenance release containing number of bug fixes and compatible with the Axis2 1.6.2 release.



Sandesha2 1.6.2 is a maintenance release that is compatible with the Axis2 1.6.2 release.



The new versions are available for download at the following locations:



http://axis.apache.org/axis2/java/core/download.cgi

http://axis.apache.org/axis2/java/rampart/download/1.6.2/download.cgi

http://axis.apache.org/axis2/java/sandesha/download.cgi



As always, we welcome any and all feedback at:



java-dev@axis.apache.org - for developer-related questions/concerns 

java-user@axis.apache.org - for general questions, usage, etc.





NOTE  - Axis2 1.6.2 compatible with Axis2 Transports 1.0.0 version











by Sagara Gunathunga at May 04, 2012 04:47 AM

May 03, 2012


Sagara Gunathunga—Code generation and WSDL exposed over HTTPS




In one of my previous post I explained how to use WSDL2JAVA tool  behind a proxy together with basic authentication. Another such  useful scenario is code generation for a WSDL which exposed over HTTPS protocol.  In this case we have to provide key store details to the WSDL2JAVA tool. Basically it expect following two Java system properties.



 javax.net.ssl.trustStore  
 javax.net.ssl.trustStorePassword  
Once you know those properties you can use WSDL2JAVA tool as follows.

    
 wsdl2java.sh -uri https://localhost:8443/services/SimpleService?wsdl -Djavax.net.ssl.trustStore=path/keystorename.keystore -Djavax.net.ssl.trustStorePassword=keypassword  





by Sagara Gunathunga at May 03, 2012 01:21 PM


Yumani Ranaweera






by Yumani Ranaweera at May 03, 2012 11:32 AM


Tharindu Mathew—Using Google collections’ MapMaker to quickly build a cache
Building a cache is standard practice in most programs to reduce the overhead of re-creating expensive objects. Using a concurrent Hash Map to build a thread safe cache is a widely user practice.
Even though each method call of concurrent Hash Map is thread safe, it is cumbersome to re-do the necessary operations. Let me list these down:

Compound operations of checking and inserting data into a HashMap – This can result in duplicate objects being created. You may have to use extensive checks using putIfAbsent to avoid these sort of situations.
Cache expiry – Getting a scheduled thread to clear the cache
Look at whether you want strong or weak references, strong or weak keys
Limit as to how many concurrent updates are allowd

Let me introduce MapMaker, a very convenient Factory object present in the Google Collections library. It lets me do everything mentioned in the above list in a single object initialization.
ConcurrentMap<Key, HeavyObject> graphs = new MapMaker()
 .concurrencyLevel(10)
 .softKeys()
 .weakValues()
 .expiration(15, TimeUnit.MINUTES)
 .makeComputingMap( new Function<Key, HeavyObject>() { 
 public Graph apply(Key key) { 
 return createHeavyObject(key); 
 } 
});


Now, all you have to do to use the cache is do a call to the get method:
graphs.get(someKey);
This will create the object, making sure other threads are blocked until the Heavy Object is created properly and the same object is returned for the specific key to any other thread as well.
Note: Make sure you implement the equals() and hashCode() methods if your Key is a complex object.
Hope you find it useful in your Java code.

         
by Mackie Mathew at May 03, 2012 06:18 AM


Srinath Perera—Scaling Distributed Queues: A Short Survey
Following is a part of the related works survey from the paper, "Andes: a highly scalable persistent messaging system". However, following has nothing about Andes, but only how different distributed queue implementations work. I will write about Andes later. 

   What is a Distributed Queue?
Distributed Queue is a FIFO data structure that is accessed by entities in a distributed environment. Working of a distributed queue will be as follows. 
There are two types of users (publishers and subscribers)
A users creates a queue (or queues may be created on demand)
Subscribers subscribe to a queue
Publisher send a message (publish) to the queue
Published message is sent to a one of the publishers who has subscribed to the queue
Distributed Queues provides strict or best effort support for in-order delivery where subscribers receives messages at the same order they have been published. (It is very hard to enforce this across all subscribers, and therefore, often implementations enforce this within each subscriber. For example, if messages m1, m2 .. m100 are published in order, each subscriber will see a subset of messages in ascending order. But there are no guarantee about the global order seen across subscribers).
   What does Scaling Distributed Queues means?
Scaling is handling larger workload by adding more resources. Workload can be increased in many ways, and we call those different dimensions of scale. There are three main dimensions.
Scaling to Handle large number of queues
Scaling to handle a queue that has large workload
Scaling to handle large messages
   Distributed Queue Implementations
There are many distributed queue implementations in JMS servers like ActiveMQ, HorentMQ etc. Focus of our discussion is that how can they scale up.

There are four choices
Master-Salve topology – queue is assigned to a master node, and all changes to the queue are also replicated to a salve node. If the master has failed, the slave can take over. (e.g. Qpid and ActiveMQ, RabbitMQ).
Queue Distribution - queues are created and live in a single node, and all nodes know about all the queues in the system. When a node receives a request to a queue that is not available in the current node, it routes the request to the node that has the queue. (e.g. RabbitMQ)
Cluster Connections – Clients may define cluster connections giving a list of broker nodes, and messages are distributed across those nodes based on a defined policy (e.g. Fault Tolerance Policy, Load Balancing Policy). It also supports message redistribution, which means if the broker does not have any subscriptions, the messages received by that broker are rerouted to other brokers that have subscriptions. It is worth nothing that server side (brokers) plays a minor role in this setup.
Broker networks - The brokers are arranged in a topology, and subscriptions are propagated through the topology until messages reach a subscriber. Usually, this uses Consumer priority mode where brokers that are close to the point of origin are more likely to receive the messages. The challenge is how to load balance those messages. (e.g. ActiveMQ)
Replication in distributed queues is inefficient as delivering messages in-order needs replication of state immediately.

In cluster connections and broker networks, in order message delivery provides a best effort guarantee only. If a subscriber has failed or subscription has been deleted, the broker nodes are force to either drop the message or to redistribute them out of order to the other brokers in the network.

Any of the above modes do not handle scaling for large messages
   Summery
Topology Pros Cons Supporting Systems
Master Slave Support HA No Scalability Qpid, ActiveMQ, RabbitMQ
Queue Distribution Scale to large number of Queues Does not scale for large number of messages for a queue RabbitMQ
Cluster Connections Support HA Might not support in-order delivery Logic runs in the client side takes local decisions. HorentMQ
Broker/Queue Networks Load balancing and distribution Fair load balancing is hard ActiveMQ

by Srinath Perera at May 03, 2012 05:15 AM

May 02, 2012


Prabath Abeysekera—Query UDTs(User Defined Types) with WSO2 Data Service Server
In this brief tutorial I ll be guiding you through the process of developing a simple dataservice which is capable of retrieving an Oracle UDT (User Defined Type) from a database using WSO2 Data Services Server.

First, login to your preferred Oracle database via your favourite SQL client tool or the sqlplus command line utility and run the following script. This will create necessary UDT structures, a table to store the UDT type as well as some sample data which will later be queried via dataservice.



Next, download the latest version of WSO2 Data Service Pack from here and extract it to a proper location in your file system. Let's call it DSS_HOME. Then copy the Oracle JDBC jar downloaded from here to DSS_HOME/repository/components/lib directory.

Now we're done with preparing the surroundings for creating the dataservice.

Let's now start the WSO2 Data Service Server and start building up the dataservice. (You can find more detailed information about developing a simple dataservice from here.). Depicted below is a sample dataservice descriptor file (.dbs) that carries a dataservice queries for retrieving and inserting UDT values in customer_address column of the customer_tbl table.

  
NOTE: 
If you carefully look at the input mappings defined for the dataservice query "q1", the UDT attributes that are being retrieved are specified in the format of "database_column_name[UDT_attribute_index]" 

Once you deploy the "UDTSample" dataservice in the WSO2 Data Services Server it will be displayed under the service list. 









You can then click on the Tryit  client functionality using which you will be able to test the data  service operations that manipulate the aforementioned UDT structures. 


I trust this simple tutorial helps you understand the basics of manipulating UDTs with WSO2 Data Services Server.  Further, I'm hoping to come up with more complex samples explaining scenarios such as how to retrieve UDTs as OUT parameters of stored procedures, how to query SQL Arrays via stored procedures/ordinary SQL queries/Ref cursors, etc soon.

by Prabath Abeysekara at May 02, 2012 07:49 PM


Srinath Perera—Authentication and Authorization Choices in WSO2 Platform
Following diagram come out of a chat with Prabath, and it shows most of the public APIs of WSO2 Identity Server, and typical design and deployment choices with implementing authentication and authorization with WSO2 platform. 

Authentication and Authorization Choices in WSO2 Platform


Each server in the WSO2 platform is built using the Carbon platform. We use the term “Carbon server” to denote any Carbon based server like ESB, AS, BPS. 

Techniques explained here are applicable across most of the WSO2 products. In the following figure, and the circles with branching out paths shows different options. 

As shown by the figure, Carbon server may receive two types of messages: messages with credentials (like passwords), and messages with tokens. When a server receives a message with credentials, the server first authenticates the request and optionally authorizes the action. When the server receives a message with tokens, generally there is no authentication step, and the token is directly validated against permissions and request is either granted or denied. 
     Authentication
Authentication needs a User store that holds the information about users and “Enforcement Point” that verifies the credentials against the User store. 
Carbon Servers support two user stores.
Database based user store
LDAP based user store
It is a common deployment pattern for multiple carbon servers in a single deployments to point to the same user store, and this provide a single point to control and manage the users. 

We can configure any Carbon server to authenticate any incoming requests. It supports many options like HTTP Basic Authentication over SSL for HTTP, WS-Security User Name Tokens, Web SAML SSO etc. This authentication is done against the users that reside the user store. 

Also, each Carbon server has a Web Service called Authentication Admin Web Service, which exposes the authentication as a Web Service to the outside. The client can invoke the Authentication Admin Web Service and get a HTTP Cookie after logging in and reuse the Cookie to do authenticated calls to a Carbon Server. 
     Authorization
In Authorization Scenarios, Carbon server receives a request that is generally already authenticated or a request that include a token. In either case, we want to check weather the authenticated user have enough permission to carry out a given action. 
Using XACML terminology, we can define three roles in such a scenario. (XACML includes other roles, which we will ignore on this discussion). 
PEP (Policy enforcement Point) intercepts requests and makes sure only authorized requests are allowed to proceed. 
PDP (Policy definition Point) stores the permissions and verify that given user have enough permissions
PAP (Policy Administration Point) let users define and change permissions. 
Carbon servers support the Policy Enforcement Point (PEP) role using a WSO2 ESB Mediator or Apache Axis2 Handler or through a custom user code. 
For Policy Definition Point (PDP), we support three ways to define permissions. 
Database based permission stores – permissions are stored in the Database
XACML – permissions are described using XACML specification
KDC - Permissions are provided as Kerberos Tokens
We support policy administration (PAP) through WSO2 Identity Server, which enables users to edit the permission definitions through the management console. 
These gives rise to several scenarios
If the Database based permission store is used, we can configure any Carbon Server to connect to the permission database directly and load the permissions to the memory. Then it authorizes user actions using those permission descriptions. Carbon servers also have an Authorization Admin Web Service that let users check for permissions of a given user remotely. 
If XACML based authorizations are used, there must be an Identity Server that acts as a PDP (Policy Definition Point). Each Carbon server (acting as the PEP, Policy Enforcement Point) invokes an Entitlement Service available in the Identity Server to check the permissions. Entitlement service is available as a Web Service or a Thrift Service. 
If Carbon server receives a Kerberos token, it talks to a configured Kerberos Server and verifies token. WSO2 IS come bundled with Apache KDC out of the box. 
More information out WSO2 Identity Server can be found from http://wso2.com/products/identity-server/ and if there are any missing features, please drop a note to WSO2 architecture List.      

by Srinath Perera at May 02, 2012 06:20 PM


Chris Haddad—Progress Sonic to Exit Middleware Market
In John Rymer’s recent blog post, Progress Software Lowers Its Sights, he breaks the news that Progress is divesting perceived ‘non-core’ middleware products.  On the selling block are Progress Sonic ESB, Savvion BPM, Actional services management, and FuseSource.  Progress’ recent strategy shift places Sonic ESB, Sonic MQ, and FuseSource implementations at risk of obsolescence.  The list of probable acquirers could mean product termination and forced migration.

John advises IT groups to:
“approach Progress as a specialist vendor with three distinct products, not an enterprise platform provider. Progress has good products, but clients must include in their evaluations of those products the continuing business execution risk that the company will face during the next year. Progress is still in transition.”
I’m not quite sure what type of IT infrastructure provider Progress will be after the transition.  Neil Ward-Dutton reviewed Progress’ strategy and decrees:
“if we take this at face value (and that’s all I can do at this stage) that means no more technology to support customers looking to implement BPM, SOA, or MDM.”
In contrast, WSO2 is a 100% open enterprise platform provider with a dedicated focus on delivering a complete, composable, and cohesive middleware platform spanning data to screen.   Our WSO2 Enterprise Platform enables SOA, BPM, API management, web application development, and Cloud.
Our WSO2 ESB, WSO2 Governance Registry, WSO2 Business Activity Monitor, and WSO2 Identity Server provide a production proven, high performance SOA middleware foundation.   We welcome you review our case studies and learn how WSO2 ESB processes more than 1 billion transactions per day for eBay, streamlines the development and maintenance of smart power grids, supports T24 core banking systems, and enables consolidated reporting across enterprise applications.  I have created an ESB evaluation framework which may assist your vendor selection.
Our Platform as a Service offering, WSO2 Stratos, delivers 13 middleware services (i.e. application server, mashup server, gadget server, business process server, business rules server, business activity monitoring, complex event processing, enterprise service bus, governance registry, identity management, data services server, relational storage service, Cassandra storage server) in a multi-tenant, on-demand form factor.  IT organizations may deploy WSO2 Stratos in an on-premise private cloud, on Amazon AWS, or through Cloud PaaS providers.
Our Offer to Progress Sonic ESB and Progress FuseSource Customers
WSO2 desires to assist Progress Sonic ESB and Progress FuseSource customers choose a viable, stable, and supported middleware platform.  We are offering free Evaluation Support to current Progress Sonic ESB and Progress FuseSource customers, and would be pleased to demonstrate how our market leading WSO2 Enterprise Service Bus and WSO2 SOA Platform meets your evaluation criteria.  Feel free to contact us via our contact form or send us an email note.
Case Studies
eBay uses 100% open source WSO2 ESB to process more than 1 billion transactions per day
eBay is the world’s largest online marketplace. WSO2 ESB helps to ensure high performance and 24×7 availability during peak holiday shopping season.
 
WSO2 ESB-Enabled Integration and Mediation Energize Smart Power Grids
Read how Dongfang Electronics Co. Ltd., one of the largest electric power automation system manufacturers in China, is using the open source WSO2 Enterprise Service Bus (WSO2 ESB) to streamline the development and maintenance of smart power grids
 
WSO2 Middleware Ensures Alfa-Bank a Promising Future in SOA
The WSO2 Application Server together with the WSO2 ESB has enabled Alfa-Bank to implement seamless integration of the T24 core banking system with its existing internal and third party systems
 
Concur Streamlines Corporate Reporting With WSO2 Open Source SOA Middleware
This case study highlights how one customer successfully implemented a SOA initiative to deliver consolidated reporting from enterprise applications distributed across multiple systems.
 
by haddadcblg at May 02, 2012 03:23 PM


Sagara Gunathunga—How easy to test your web service over HTTPS



I have seen many times people having issues with testing web services over HTTPS. This is not an unexpected behavior because number of improvements have been introduced from Axis2 1.5 release for HTTPS transport. With older versions it is possible to use HTTPS transport with Simple HTTPServer and some people already familiar with it too, but recent Axis2 releases no longer support for this option and it is compulsory to use Servlet transport  in order to enable HTTPS.



Above modifications doesn't effect much on production systems because it's always recommend to use an Application server for production systems. When it come to development  stage this changes required to use a Servlet container to test HTTPS services, Apache Tomcat is one of the good choice for this.



In this post I will discuss how to use Maven Jetty Plug-in to test Axis2 HTTPS transport very easily, all you need is to follow few steps that I have given below.



Step -1  Configure Axis2 for HTTPS transport.



It is required to define AxisServletListener in your axis2.xml for HTTPS transport, in case  if you want to use both HTTP ad HTTPS it is possible to define AxisServletListener as two entries with two ports this guide provide more details about this. If you don't have axis2.xml file copy it from binary distribution and replace existing  " transportReceiver " with following entries.



  <transportReceiver name="http"  
     class="org.apache.axis2.transport.http.AxisServletListener">  
     <parameter name="port">8080</parameter>  
   </transportReceiver>  
   <transportReceiver name="https"  
     class="org.apache.axis2.transport.http.AxisServletListener">  
     <parameter name="port">8443</parameter>  
   </transportReceiver>   



Step - 2  Generate KeyStore.



Usually this is kind of a time wasting task but fortunately you can use keytool-maven-plugin to auto-generate keystore in each run. The only required step is add following entries into your POM file and it will save your time a lot.



       <plugin>  
         <groupId>org.codehaus.mojo</groupId>  
         <artifactId>keytool-maven-plugin</artifactId>  
         <executions>  
           <execution>  
             <phase>generate-resources</phase>  
             <id>clean</id>  
             <goals>  
               <goal>clean</goal>  
             </goals>  
           </execution>  
           <execution>  
             <phase>generate-resources</phase>  
             <id>genkey</id>  
             <goals>  
               <goal>genkey</goal>  
             </goals>  
           </execution>  
         </executions>  
         <configuration>  
           <keystore>${project.build.directory}/jetty-ssl.keystore</keystore>  
           <dname>cn=localhost</dname>  
           <keypass>axis2key</keypass>  
           <storepass>axis2key</storepass>  
           <alias>axis2key</alias>  
           <keyalg>RSA</keyalg>  
         </configuration>  
       </plugin>  




Spacial Note - Above approach is not an replacement for production server configurations . It is highly recommend to follow standard procedures to configure production servers for HTTPS.



Step - 3  Configure and Run the service.



Now you need to add SSL Connectors for the Jetty configuration by adding following entries.



  <plugin>  
         <groupId>org.mortbay.jetty</groupId>  
         <artifactId>maven-jetty-plugin</artifactId>  
         <configuration>  
           <webAppConfig>  
             <contextPath>/</contextPath>  
           </webAppConfig>  
           <connectors>  
             <connector  
               implementation="org.mortbay.jetty.security.SslSocketConnector">  
               <port>8443</port>  
               <maxIdleTime>60000</maxIdleTime>  
               <keystore>${project.build.directory}/jetty-ssl.keystore</keystore>  
               <password>axis2key</password>  
               <keyPassword>axis2key</keyPassword>  
             </connector>  
             <connector  
               implementation="org.mortbay.jetty.nio.SelectChannelConnector">  
               <port>8080</port>  
               <maxIdleTime>60000</maxIdleTime>  
             </connector>  
           </connectors>  
         </configuration>  
       </plugin>  



Run "mvn Jetty:run " this will start jetty server with both HTTP and HTTPS connectors . Now you can access to the  WSDL content over HTTPS. According to above example you can fine WSDL file here.





Step- 04 - Configure  Client.



Since now you can access to the WSDL file you are free to use any of your client side preferences to create a WS client , you may generate stub or possible to write service/operation clients. Before you invoke your service you need to perform one more extra step, that is provide your key store details to Java run time . In simply we can use above generated keystore with our client too.



Add following two lines before you invoke your client.



 System.setProperty("javax.net.ssl.trustStore","[location]/jetty-ssl.keystore");  
 System.setProperty("javax.net.ssl.trustStorePassword", "axis2key");  





You can download full source code for this sample from here.























by Sagara Gunathunga at May 02, 2012 10:31 AM

May 01, 2012


Kathiravelu Pradeeban—Digging into addictions


I often have felt 
sorry for the people who do routine tasks as jobs, which doesn't involve
 creative thinking or some change from a predefined agenda. Though this 
relates to the clerical jobs, call centers, or the jobs where bulk of 
employees are considered a cheap labour carrying forward the order of a 
big-guy. In late years with the rise of IT, many concerns were risen 
considering the fact that programmers are considered the same. Human nature is to avoid repetition of uninteresting tasks, with the exception of something that is addictive. An addict may find something interesting, which may not be for others. Someone often gets addicted to something, when he wants to be distracted
 from his mainstream life. The addiction can be the excessive use of 
alcohol, drugs, porn, or whatever. Addiction provides a short-term 
relief from the pain and the pressure of the real life (let's call it 
the first life). It leads to a feeling of guilty, inefficiency,
 and low self esteem.
 This vicious cycle continues. However, mild-addictions such as addiction to music or movies may not be harmful at all.




Facing the problem and trying to solve it
 instead of finding distractions to run away from it might be the ideal 
solution for this. If that is something difficult to achieve, and if 
there is a real need to keep you distracted from the issue, I recommend 
getting involved in other healthy activities instead. When we get even a mild fever, we seek the assistance of medicine and the doctors. We always tend to underestimate the health of the mind. We always take leaves for physical illness, whilst silently ignoring the wounds of the heart. If you ask me, taking a leave for being depressed is perfectly fine. As I might have mentioned somewhere before too, it is an ill-condition of the mind. Mental health is equally or even more important than that of physical.




If you are into 
software development, open source projects give you a second life. I 
would encourage you to contribute. Blogging or creating a positive 
online presence would suit for everyone. One can be completely or 
partially anonymous online, will contributing positively to the world 
knowledge.

by Kathiravelu Pradeeban at May 01, 2012 05:29 PM


Sagara Gunathunga—Axis2 clustering on Tomcat




In this post I will discuss how to setup a Axis2 cluster using two Tomcat servers.  It's not my intention to describe Axis2 cluster architecture or cluster configuration language, if you need such details refer provided references at the end of this post. Axis2 cluster implementation is based on pure API which you can implement using any Java multicast communication framework, by default Axis2 provide a cluster implemantation based on Tomcat Cluster Communication Module also known as Apache Tribes. Following diagram illustrate the design we are going to discuss in this post but I skip load balancer setup for simplicity but in real world scenario you could use load balancer such as Apache2 server or WSO2 Load Balancer that support for more advanced options. 








Pre-requirements   




1. Apache Tomcat server. ( Version 7.x.x preferable. )

2. Apache Axis2 WAR distribution (Version 1.6.2 preferable.)








Here I use same machine to setup two Tomcat server instances hence it's required to change server configuration of one instance. Let's say Node-1 having default configuration and Node-2 having custom configuration. Open the server.xml file and change server port, port nubers of HTTP and AJP Connectors as follows.    




Node-1 (default configuration)

 <Server port="8005" shutdown="SHUTDOWN">  
 <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" />  
 <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />  





Node-2 
 <Server port="9005" shutdown="SHUTDOWN">  
 <Connector port="9090" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="9443" />  
 <Connector port="9009" protocol="AJP/1.3" redirectPort="9443" />  





Now install Axis2 WAR distribution on both server instances and stop both servers to edit axis2 configuration file. Open axis2.xml file and edit cluster settings as follows. 





 Node -1




     <parameter name="AvoidInitiation">false</parameter>   
     <parameter name="domain">sample.cluster.domain</parameter>  
     <parameter name="mcastBindAddress">127.0.0.1</parameter>  
     <parameter name="localMemberHost">127.0.0.1</parameter>  
     <parameter name="localMemberPort">4000</parameter>  








  Node -2 



     <parameter name="AvoidInitiation">false</parameter>   
     <parameter name="domain">sample.cluster.domain</parameter>  
     <parameter name="mcastBindAddress">127.0.0.1</parameter>  
     <parameter name="localMemberHost">127.0.0.1</parameter>  
     <parameter name="localMemberPort">4001</parameter>   





Note that in a real network setup it's required to edit mcastBindAddress, localMemberHost settings in addition to  localMemberPort value but in my local machine only localMemberPort has changed.  After this has done start the Node -1, if there is no issue in your setup you could able to see log messages as follows on node-1 server console. 



 [INFO] Initializing cluster...  
 [INFO] Cluster domain: sample.cluster.domain  
 [INFO] Using multicast based membership management scheme  
 Apr 30, 2012 6:37:13 PM org.apache.catalina.tribes.transport.ReceiverBase bind  
 INFO: Receiver Server Socket bound to:/127.0.0.1:4000  
 Apr 30, 2012 6:37:13 PM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket  
 INFO: Setting cluster mcast soTimeout to 500  
 Apr 30, 2012 6:37:13 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers  
 INFO: Sleeping for 1000 milliseconds to establish cluster membership, start level:4  
 Apr 30, 2012 6:37:14 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers  
 INFO: Done sleeping, membership established, start level:4  
 Apr 30, 2012 6:37:14 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers  
 INFO: Sleeping for 1000 milliseconds to establish cluster membership, start level:8  
 Apr 30, 2012 6:37:15 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers  
 INFO: Done sleeping, membership established, start level:8  
 [WARN] Local member advertising its IP address as 127.0.0.1. Remote members will not be able to connect to this member.  
 [INFO] Local Member 127.0.0.1:4000(sample.cluster.domain)  
 [INFO] No members in current cluster  
 [INFO] Cluster initialization completed.  
 Apr 30, 2012 6:37:15 PM org.apache.catalina.startup.HostConfig deployDirectory  
 INFO: Deploying web application directory /home/sagara/dev/servers/axis2-clustering/tomcat/node1/webapps/ROOT  
 Apr 30, 2012 6:37:15 PM org.apache.coyote.AbstractProtocol start  
 INFO: Starting ProtocolHandler ["http-bio-8080"]  
 Apr 30, 2012 6:37:15 PM org.apache.coyote.AbstractProtocol start  
 INFO: Starting ProtocolHandler ["ajp-bio-8009"]  
 Apr 30, 2012 6:37:15 PM org.apache.catalina.startup.Catalina start  
 INFO: Server startup in 3068 ms  









Now start the Node -2 and monitor log messages on Node-2 console 

 [INFO] Initializing cluster...  
 [INFO] Cluster domain: sample.cluster.domain  
 [INFO] Using multicast based membership management scheme  
 Apr 30, 2012 6:37:50 PM org.apache.catalina.tribes.transport.ReceiverBase bind  
 INFO: Receiver Server Socket bound to:/127.0.0.1:4001  
 Apr 30, 2012 6:37:50 PM org.apache.catalina.tribes.membership.McastServiceImpl setupSocket  
 INFO: Setting cluster mcast soTimeout to 500  
 Apr 30, 2012 6:37:50 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers  
 INFO: Sleeping for 1000 milliseconds to establish cluster membership, start level:4  
 [INFO] New member 127.0.0.1:4000(sample.cluster.domain ) joined cluster.  
 Apr 30, 2012 6:37:51 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers  
 INFO: Done sleeping, membership established, start level:4  
 Apr 30, 2012 6:37:51 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers  
 INFO: Sleeping for 1000 milliseconds to establish cluster membership, start level:8  
 Apr 30, 2012 6:37:51 PM org.apache.catalina.tribes.io.BufferPool getBufferPool  
 INFO: Created a buffer pool with max size:104857600 bytes of type:org.apache.catalina.tribes.io.BufferPool15Impl  
 Apr 30, 2012 6:37:52 PM org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers  
 INFO: Done sleeping, membership established, start level:8  
 [WARN] Local member advertising its IP address as 127.0.0.1. Remote members will not be able to connect to this member.  
 [INFO] Local Member 127.0.0.1:4001(sample.cluster.domain )  
 [INFO] Members of current cluster  
 [INFO] Member1 127.0.0.1:4000(sample.cluster.domain )  
 [INFO] Trying to send initialization request to 127.0.0.1:4000(sample.cluster.domain )  
 [INFO] Received configuration initialization message  
 [INFO] Trying to send initialization request to 127.0.0.1:4000(sample.cluster.domain )  
 [INFO] Received state initialization message  
 [INFO] Cluster initialization completed.  
 Apr 30, 2012 6:37:52 PM org.apache.catalina.startup.HostConfig deployDirectory  
 INFO: Deploying web application directory /home/sagara/dev/servers/axis2-clustering/tomcat/node2/webapps/ROOT  
 Apr 30, 2012 6:37:52 PM org.apache.coyote.AbstractProtocol start  
 INFO: Starting ProtocolHandler ["http-bio-9090"]  
 Apr 30, 2012 6:37:52 PM org.apache.coyote.AbstractProtocol start  
 INFO: Starting ProtocolHandler ["ajp-bio-9009"]  
 Apr 30, 2012 6:37:52 PM org.apache.catalina.startup.Catalina start  
 INFO: Server startup in 13334 ms   









Additionally now you should able to see following log messages on Node-1 server console. 

 Apr 30, 2012 6:37:50 PM org.apache.catalina.tribes.io.BufferPool getBufferPool  
 INFO: Created a buffer pool with max size:104857600 bytes of type:org.apache.catalina.tribes.io.BufferPool15Impl  
 [INFO] New member 127.0.0.1:4001(sample.cluster.domain) joined cluster.  
 [INFO] Received GetConfigurationCommand initialization request message from 127.0.0.1:4001(sample.cluster.domain)  
 [INFO] Received GetStateCommand initialization request message from 127.0.0.1:4001(sample.cluster.domain) 







If you have followed me up to this point you have successfully setup a Axis2 cluster with two nodes. Now you can deploy any cluster aware web service on this cluster. For testing purposes let's write following POJO service and will deploy on both servers. I have given service code and service.xml  below. 



 package sample;  
 import org.apache.axis2.context.MessageContext;  
 public class Count {  
   public int count() {  
     int count;  
     MessageContext mc = MessageContext.getCurrentMessageContext();  
     Object ob = mc.getConfigurationContext().getProperty("count");  
     if (ob == null) {  
       count = 1;  
     } else {  
       count = (Integer) ob;  
       count++;  
     }  
     mc.getConfigurationContext().setProperty("count", count);  
     return count;  
   }  
 }  




Note that we use  ConfigurationContext to store our count values, ConfigurationContext is replicate among cluster members.





 <service name="count">   
   <parameter name="ServiceClass">sample.Count</parameter>  
   <operation name="count">  
   <messageReceiver class="org.apache.axis2.rpc.receivers.RPCMessageReceiver" />  
   </operation>  
 </service>  





In order to test our cluster aware service we need a Web service client, for the simplicity I will use a browser and Axis2 REST support.  Since we skip the load balancer setup following two different URLs can be use to invoke services . Once you invokes services on both servers you can notice that the count value is shared among two Axis2 instances clearly.




 http://localhost:8080/axis2/services/count/count  
 http://localhost:9090/axis2/services/count/count   





References


Introduction to WSO2 Carbon Clustering.
WSO2 Carbon Cluster Configuration Language.
Enabling Apache Axis2 Clustering.












by Sagara Gunathunga at May 01, 2012 11:29 AM


Kathiravelu Pradeeban—Apache Meetup, Kandy and Community Matters!

Yesterday (28th April, 10 a.m. - 4.30 p.m.) we had an Apache Meetup at the Engineering Faculty of the University of Peradeniya. It was an excellent event, which followed the recent Apache BarCamp, Colombo. I presented how to build a new open source community for the audience. The presentation was titled, "Community Matters!!!111" and was based on "Community Matters," the GSoC Mentor Summit discussion that I coordinated.


If the presentation below takes a long time to load, feel free to download it!


 Community Matters!!!111   View more presentations from Kathiravelu Pradeeban 
 

by Kathiravelu Pradeeban at May 01, 2012 10:27 AM

April 29, 2012


Chris Haddad—VMWare’s Linux in the Cloud Strategy
VMWare’s Linux in the Cloud strategy is only half the solution for Cloud applications.  Development teams also require a Cloud Native application platform.

While many Linux distributions include a basic application server, Linux by itself does not deliver a complete application platform.   At a minimum, development teams commonly layer WebSphere, JBoss, Tomcat, or WSO2 Carbon Application servers on top of Linux.  VMWare Cloud Foundry’s open architecture will front-end any application server through their Droplet Execution Agent (DEA) architecture (after integration work is performed!).  Derek Collison has authored material describing the ability to plug-in third-party application servers.  The level of integration between Cloud Foundry’s framework and the application platform will differentiate offerings.
VMWare’s recent announcement equating VMWare Cloud Foundry with Linux demonstrates an operating system focused mindset, and does not address the environment required for developing enterprise applications.  As mentioned by the Cloud Foundry team, most of their work is below the waterline and does not directly address innovation relevant to application developers.
As to whether Cloud Foundry is a cloud operating system or capable of building vendor-neutral clouds, I defer to Charles Babcock’s article and your experience.  Is your team building an application server agnostic Platform as a Service environment on top of Cloud Foundry?   What limitations do you experience when wrapping traditional application server clusters with a PaaS framework (e.g. VMWare Cloud Foundry)?
 
 
by haddadcblg at April 29, 2012 05:08 PM


Chris Haddad—Know your Cloud Dimensions
Summary:  Before selecting PaaS infrastructure, understand how sharing, location, and responsibility impact your public/private Cloud and internal/external Cloud decision.

My colleague Gary Hein once mentioned, “It seems there are only 500 guys in IT that you constantly run across.”  Over the past four years, I have formed a relationship with Sinclair Schuller, Apprenda’s CEO.  Sinclair has been recently turning up the marketing amp and conveying the benefits/drawbacks of private and public cloud.  Sinclair’s views can be found in an article recently published on CIO.com and a VMBlog Q&A session.  Both the article and Q&A session are worthy reads.  If you enjoy watching videos, I have created a Cloud dimension video with Jonathan Marsh.
As a fellow private cloud arms dealer, I am pleased Sinclair promotes Platform as a Service and private use cases. According to the CIO.com article,
“Private PaaS is the deployment of a PaaS software layer on an enterprise’s internal infrastructure with the goal of exposing the PaaS service to the developers within an enterprise’s various lines of business.”
While the statement is accurate, I wish Sinclair had expounded on the value derived from using PaaS hosted on private external Clouds.  Public/private and internal/external are two separate dimensions.  Public, private, or community attributes specify how widely the cloud service is shared; a sharing dimension.  Internal or external denote the consumer’s view of the Cloud’s service interface.  The view is associated with a consumer’s responsibility for service development, operations, and management; a responsibility dimension.  A third dimension, on-premise or outsourced, describes where the service assets are located; a location dimension.  Many architects conflate the three dimensions. NIST has recently published a Cloud Computing Reference Architecture which spends considerable prose disentangling the concepts.  According to NIST,
“A private cloud gives a single Cloud Consumer‟s organization the exclusive access to and usage of the infrastructure and computational resources. It may be managed either by the Cloud Consumer organization or by a third party, and may be hosted on the organization‟s premises (i.e. on-site private clouds) or outsourced to a hosting company (i.e. outsourced private clouds).”
Let’s run through three quick use cases describing public, private, and community:

A public cloud service is accessible to any consumer.  For example, all organizations who have sales teams.
A private cloud service is accessible to only members of a single team. For example, a custom tailored Enterprise Resource Planning application delivered as a service to company employees.
A community cloud blends the two access models. A community cloud service is accessible to a select, exclusive group. For example, a classified information service delivered to government agencies

A person or organization will often use and deliver cloud services across private, public, and community environment.  A hybrid cloud strategy delivers, spans, and connects clouds across all dimension attributes.  According to NIST,
“A hybrid cloud is a composition of two or more clouds (on-site private, on-site community, off-site private, off-site community or public) that remain as distinct entities but are bound together by standardized or proprietary technology that enables data and application portability.”
To effectively implement a hybrid cloud, the solution must exhibit interoperability and policy federation across cloud services.  Interoperability and federation are two difficult to implement concepts. Teams should choose technologies such as XACML, OAuth, SAML, JSON, RESTful interfaces.  Cloud blending using technologies alone is often difficult.  Infrastructure products and services such as WSO2 Stratos Identity as a Service, WSO2 Stratos Governance as a Service, and WSO2 Stratos Data as a Service can assist the distillation process (see Figure 1)
Figure 1. WSO2 Stratos Platform as a Service

 
New Cloudy infrastructure is often required to build internal clouds.  When you are the service provider, the cloud is internal to your team. Your team sees all the complexity, dependencies, and inner relationships. Internal clouds require your team to be experts in demand management, capacity management, resource monitoring, resource management, deployment automation, billing, and scalability tuning practices.  According to Sinclair,
“PaaS is a software layer that typically stitches together networked resources including OS instances, database server instances, web server instances, and even load balancers into a single, shared logical hosting layer. Essentially, PaaS is best summarized as a data center OS.”
Private PaaS infrastructure attempts to turn your team into a data center OS service consumer.  Your team should only sees a simple, easy to use service interface (see Figure 2).
Figure 2: Encapsulating capabilities behind a simple, easy to use service interface
Encapsulating a capability within a service 
The service interface hides complex technology and processes required to deliver an elastic and scalable cloud on shared resources.  However, if the service interface is leaky, you must still contend with complexity (see Figure 3).
Figure 3: Example complexity exposed by Leaky PaaS
A PaaS Leaking Complex Infrastructure Details
We find certain Paas offerings both hide complexity and provide a solid, easy to use service interface, or they provide a difficult to use leaky service interface.  A leaky PaaS service interface:

Exposes machine host names instead of URLs
Specifies confining Java Virtual Machine memory configuration limits instead of delivering an elastic and scalable memory pool
Requires server reboots to scale compute clusters
Exposes tenants to security and Quality of Service risk

A leaky PaaS service interface requires developers to

Download and install application platform modules instead of subscribing to application platform services
Modify load balancer tables instead of specifying service policy
Deploy applications via system administration console commands

Our strategic goal at WSO2 is to deliver on-premise Cloud middleware and outsourced Cloud middleware services that enable your development teams to more effectively design, develop, deploy, and manage your applications.  Your development teams should not care about network routing, machine counts and size, Java Virtual Machine configurations, or clustering protocols.  Your teams are free to focus on provisioning services, declaring policies, defining registry spaces, and building/enhancing business domain capabilities.   During my strategy call with Paul Fremantle, co-founder and CTO of WSO2, Paul mentioned a strategic focus to deliver a SLA based offering (rather than an infrastructure based offering) delivering tiered level of services and the ability for your teams to charge by transaction and business use rather than storate/network bytes and processing cycles.
Why do we care about public/private/community, internal/external, and on-premise/outsourced?  A Cloud service’s position on the dimensions directly impacts your responsibility and risk.

Public/private/community impacts the provider’s ability to incorporate your requirements into the service. A private service can be extensively customized and delivered on your release schedule. A public service can only be configured and is often general purpose. While a community cloud often incorporates special purpose domain capabilities.
Internal/external impacts your role and responsibility in maintaining the reliability, availability, scalability, and security of the cloud.  Are you an expert in data center operations?
On-premise/outsourced impacts whether you are responsible for the assets. Do you want to own hardware?

Tread carefully when adopting PaaS today.  Use the location, ownership, and sharing dimensions as an architecture and product selection starting point.  Table 1 maps the three dimensions to common cloud terms and Figure 4 visually illustrates the differences.  A table or visual of ‘1’ indicates ‘Low’ or ‘Near’.  For example, ‘low sharing’, ‘low responsibility’ or ‘near location’.  A table or visual value of ‘3’ indicates ‘High’ or ‘Far’.  For example, ‘high sharing’, ‘high responsibility’, or ‘far location’.
Table 1: Mapping sharing, location, and responsibility to Cloud dimensions



 Cloud terms

sharing


location


responsibility



public-external-outsourced

3


3


1



community-external-outsourced

2


3


2



private-external-outsourced

1


3


1



private-external-on-premise

1


3


2



public-internal-on-premise

3


1


3



community-internal-on-premise

2


1


3



private-internal-on-premise

1


1


3



private-internal-outsourced

1


1


2



Your requirements

?


?


?




 
Figure 4: Visual representation of Cloud Dimensions
Cloud Dimension Kiviat
 
To determine where your projects fit within the dimensions, use the following roadmap:

Determine goals and outcomes
Define acceptable risk (Data sensitivity, QoS, requirements, schedule)
Establish reasonable responsibilities for in-house team (operations, development, project management)
Determine solution specialization requirements (platform stack, business processes, rules, data, complex event processing, data)

After you determine your requirements and fit within Cloud dimensions, create a matrix to evaluate Platform as a Service offerings.  Because most individuals and organizations will require services landing across multiple landscape positions, investing in PaaS offerings that span public/private/community, internal/external, and on-premise/outsourced is desirable.   WSO2′s Carbon Enterprise Application Platform uniquely spans all environments (See Figure 5).
Figure 5: WSO2′s PaaS Deployment Choices
on-premise private PaaS, public cloud, and on-premise terrestrial
Beware of false clouds or cloud washed platforms as defined by Frank Scavo, Founder, President @ Strativa, in his blog post.  I have created a Platform as a Service Evaluation Framework to help you identify the Cloudiness quotient of your PaaS.
 
by haddadcblg at April 29, 2012 02:54 PM


Kasun Gunathilake—How to remote debug Apache Cassandra standalone server

In order to debug the cassandra server from your favorite IDE. You need to add the following into cassandra-env.sh located in apache-cassandra-1.1.0/conf directory.

JVM_OPTS="$JVM_OPTS -Xdebug"
JVM_OPTS="$JVM_OPTS -Xnoagent"
JVM_OPTS="$JVM_OPTS -Djava.compiler=NONE"
JVM_OPTS="$JVM_OPTS -Xrunjdwp:transport=dt_socket,server=y,address=5005,suspend=n"
cassandra-env.sh




After adding this, once you start the server you can see the following line printed in cassandra console

"Listening for transport dt_socket at address: 5005" 

This the port that you specified in JAVA_OPTS. You can change it to some other value as you want.

Now configure your IDE to run on debug mode.



Now you can debug the apache cassandra server from your favorite IDE :)

by Kasun Weranga Gunathilake at April 29, 2012 10:57 AM

April 27, 2012


Isuru Suriarachchi—Apache CXF support in WSO2 Application Server
WSO2 Application Server mainly supports hosting Web Services and Web Applications. One of the major features included in the upcoming release (version 4.5.0) of WSO2 AS is the integration of Apache CXF. So from AS 4.5.0 on-wards, CXF users also will be able to host their applications inside WSO2 AS with great ease. Once you deploy your CXF application inside WSO2 AS, it automatically inherits lots of benefits from the underlying WSO2 Carbon platform.

Ability to use the Carbon user store through CarbonContext
Ability to use WSO2 Registry API’s through CarbonContext
Fine grain authorization through WSO2 Identity Server features
All the benefits inherited through OSGi
Easy to use Management Console for CXF JAX-WS/JAX-RS Web Applications
Hot deployment and Hot update for your CXF Web Applications

Following are some screen shots of the UI provided by WSO2 AS for CXF webapps. You can upload CXF webapps through the management console as follows.

Once you upload your CXF webapp, you can see the list of webapps as follows.

By clicking on the “Find Services” link, you can see the list of Web Services exposed by the particular CXF webapp.
We’ve already completed this integration on WSO2 Carbon trunk and now we are looking at the possibilities of making this feature more useful. For example, we’ll be adding the “Try-It” capability into CXF services and providing statistics for each and every service. AS 4.5.0 is expected to be released by the end of June and it will contain this new feature.

         
by isurues at April 27, 2012 12:59 PM


Madhuka Udantha—JUnit Test for Beginners
This post will explain JUnit test. By following this steps you will make you first JUnit test in 5 minuets. Here are some points on JUnit test. 
     A unit test is written by a developer that executes a specific functionality in the code under test. 
    Unit tests ensure that code is working as intended even after code changes. 
    JUnit 4.x is a test framework (here we will be using 4.10) 
    Tests should not depend on other tests because JUnit assumes that all test methods can be executed in an arbitrary order
 
  Let’s create Code for JUnit
  1. Create a new project "UnitTest"
  2. Download JUnit4.1.0.jar from the JUnit website at http://www.junit.org/ . The download contains the "junit-4.*.jar" which is the JUnit library. Then Junit library will be added to our Java project.
  3. Create Java class to be test "Calculator.java"
  package org.madhuka;
  public class Calculator {     
    public int add(int x, int y) {      
        return x + y;      
    }      
}      
  4. Create Source Folder called 'test' [good practice to keep the test code separated from the regular code (src).]
  
  5. Add Java Class 'CalculatorTest' for test source folder
  package org.madhuka;
  import org.junit.Test;
  import static org.junit.Assert.assertEquals;
  public class CalculatorTest {     
    @Test      
    public void testAdd() {      
        Calculator tester = new Calculator();      
        assertEquals("Result", 5, tester.add(2, 3));      
    }      
}
 























  6. Lets run the test via Eclipse
  Right click on your new test class and select Run-As → JUnit Test.
  
  7. The result of the tests will be displayed in the JUnit View.   
[Change code such as and try it]
  
  [go to Calculator.java class and see that add() function is due subtraction not the addition, do the correction in code and run it back then It give Green Color bar with tell test run success]
  public int add(int x, int y) {     
        return x + y;      
    }
  8. Run test via code. So We will create TestRunner.java [ This class will execute your test class and write potential failures to the console.]                                   
package org.madhuka;
    import org.junit.runner.JUnitCore;       
import org.junit.runner.Result;        
import org.junit.runner.notification.Failure;
    public class TestRunner {       
    public static void main(String[] args) {        
        Result result = JUnitCore.runClasses(CalculatorTest.class);        
        for (Failure failure : result.getFailures()) {        
            System.out.println(failure.toString());        
        }        
    }        
}        
     
    
  

by Madhuka at April 27, 2012 12:09 PM


Sanjeewa Malalgoda—Tenant Aware Load Balancer Concept
Tenant Aware Load Balancer is the upcoming Load Balancer from WSO2. If a Cloud deployment has to be able to scale to thousands, hundreds of thousands or millions of tenants, we need tenant-partitioning. This means, for a single Cloud service, there will be multiple clusters, and each of these Service clusters will handle a subset of the tenants in the system.Here i will briefly  describe about it. I will add complete article on this later.


Tenants are loaded in demand and assign to a cluster
Unloading the unused tenants.
Notifying the load balancers when a tenant is loaded.
by adding the Loadbalancers in to a group communication group and publishing a message to the group when a tenant is loaded.
Can define statically tenants to each clusters
Single service may have multiple clusters and each cluster is assigned in tp particular tenant range 
Always running an additional cluster(Default cluster) to quickly load new tenants.

Following diagram describes the overall design of tenant aware load balancer



by Sanjeewa Malalgoda at April 27, 2012 07:43 AM


Ganesh Prasad—An Optimistic Approach to Identity

I've been working in the Identity Management area for a few years now, and I've seen three different industries up close (banking, insurance and telecom). What I'm struck by in all these industries is that none of them has historically been customer-centric in their business approach. For decades, banks have always looked at their customers through the prism of accounts, insurance companies through policies, and telecom companies through billing accounts and sometimes carriage services (broadband or mobile services). And everywhere, the holy grail is the same - "single view of customer". Identity and Access Management (IAM) is the way these organisations aim to achieve single view of customer as well as other benefits.

However, IAM initiatives at organisations in all these industries have generally floundered. Why?

I believe that IAM is simple but subtle. That's why although it's not hard to design and deliver an IAM system, it's also treacherously easy to get it wrong.

Some of the major reasons why organisations struggle with IAM are these:

1. Rather than bite the bullet and create a top-level data entity called "customer" with its own unique identifier, organisations choose what they consider a cheaper compromise because of a misplaced belief that using a surrogate for customer (i.e., account, policy, billing account) would somehow do the job. Reality check: it doesn't, and it's more expensive in the long run.

2. Even where identifiers are created for customers, these are not carefully designed. The result is that many identifiers that are chosen have business meaning. It's quite funny at one level to see a system designed with a person's email address as their identifier, and where the major business pain point is that it's very hard to handle the situation where a customer changes their email address. (Why are we not surprised?) Quite often in such cases, there is no other way around the problem but to delete and re-create the customer record.

3. Even where organisations avoid the first two mistakes and embark on an IAM initiative to tie customer data across multiple systems to a new, unique and meaning-free customer ID, they run into logistical problems relating to the existing user base. They struggle to "marry" records across systems to the appropriate customer entity because of the sheer volume of data involved, the cost of changing existing systems, the unreliability of matching algorithms and the need to replace engines while the plane is flying, so to speak. The two problems with matching algorithms are false positives (two or more customers being assigned the same identifier) and false negatives (a customer being assigned two or more identifiers).

I have some suggestions that can make life easier.

1. Create a database external to all existing systems that will maintain mappings. [Resist the temptation to migrate customer attributes from other systems to this one. This is just a mapping database, not a customer master. Use Master Data Management (MDM) principles instead to keep data in source systems in sync.]

2. Use a universally unique and meaning-free identifier for customers. Version 4 (random) UUIDs are a great scheme to use.

3. Adopt an optimistic model of "eventual consistency". I.e., generate a new customer UUID corresponding to every system record, in effect assuming (in the case of a bank) that each account belongs to a different customer, then pare them down to reflect known relationships. 

a) You can generate UUIDs for a system in an optimistic way because the probability of two UUIDs conflicting is infinitesimally low, even if you have hundreds of millions of customers. You can check for duplicates out of band if you're paranoid.

b) Similarly, you can optimistically generate UUIDs in a federated way (i.e., each system generates its own UUIDs corresponding to its surrogate records). The probability of conflict is so low it's worth doing this and checking for duplicates out of band.

c) You can afford to start with a system with a large number of false negatives (but no false positives) because this corresponds to a siloed organisation with no "single view of customer". False positives are a greater danger, and we avoid that with this scheme.

d) You can use the existing intelligence in your systems (i.e., the knowledge of which records belong to the same customer) to merge customer UUIDs relating to the same physical customer by eliminating all but one of them at random. Since UUIDs are meaningless, it doesn't matter which one you keep and which ones you remove.

Now you're no worse off than you were before in terms of data quality (i.e., your data is just as clean in terms of known relationships). But structurally, you're far better off because you now have a customer data entity for the first time. As your data quality improves with more reliable mappings, the siloes effectively disappear and you get to a "single view of customer" with no more changes to data structures or processes.

In the case of a telecom company, your mapping database will now consist of three parts. The first part will map customer UUIDs to billing accounts. The second part will map customer UUIDs to product holdings (mobile, broadband and other carriage services, media products, etc.) The third part will map customer UUIDs to other customer UUIDs to reflect corporate organisational structures and household relationships. With this model, the many problems that telecom companies currently face will simply melt away.

- We can see all the product holdings of a customer to determine what else to sell them. We can see this at an individual customer level as well as at the level of a household or organisational unit.
- We can sell media products even to customers who haven't purchased an underlying carriage service
- We can group billing accounts independently of product holdings. In a household, the kids use various products but mum or dad alone may pay the bill.

As you can see, this kind of design isn't hard. But it requires conceptual clarity around the nature of Identity. As I said before, IAM is simple but subtle. It isn't hard to design and deliver an IAM system, but it's treacherously easy to get it wrong.


by Ganesh Prasad at April 27, 2012 06:44 AM


Ruchith Fernando—My Poster on Private Anonymous Messaging
I presented one of my projects at the CERIAS Symposium 2012. 






This project is about a set of peers who wish to remain anonymous (even to each other) connected to a particular entity. In a situation where the common entity has limited connectivity to the peers, this research provides means of being able to distribute the messages transmitted by the common entity among all peers. 


And I'm
by Ruchith Fernando at April 27, 2012 02:10 AM

April 25, 2012


Kathiravelu Pradeeban—Google Summer of Code 2012 and AbiWord


Being a mentor for the Google Summer of 
Code with AbiWord for the second time is going to be an interesting experience once more. It was a
 nice memory going through all the 29 proposals for AbiWord 
and reviewing them as a mentor. Selected students were announced by Google on 1900UTC, 23rd of April.




The list of accepted students, along with their project proposals as well as their mentors are given below.





1) Tanya Guza - "Improve ODF support"  - Mentored by Hub

2) Kousik Kumar - "Table Improvements" - Mentored by Simon

3) Aaditya - "Implement Rotated Text" - Mentored by Martin

4) Vincent (Zuyin Kang) - "Dialog improvements" - Mentored by Pradeeban

5) Bafna - "Implement and Improve the import and export of math from/to odt, doc & docx formats" - Mentored by Jean

6) Serhatkiyak - "Improving Abiword's OpenXML(.docx) support" - Mentored by Dom

An interesting point to notice is that, since 2006, AbiWord has  
successfully been participated in all the Google Summer of Codes (2006, 
 2007, 2008, 2009, 2010, 2011, and 2012). Hence this becomes the 7th  
consecutive year for AbiWord to participate in Google Summer of Code! I 
wish the 6 students who got selected a great summer of code with 
AbiWord, and I hope they will continue to be a long term contributors 
even after their summer. At the same time, I should also note that, we 
had to miss a few nice students as we have only 6 slots. Hope they will 
still continue with their development on AbiWord.

by Kathiravelu Pradeeban at April 25, 2012 08:42 PM


Kathiravelu Pradeeban—DZone Kolamba Meetup

We had the first DZone Kolamba Meetup at 4.30 p.m - 6.30 p.m today, at WSO2. The theme was Big Data. Given below is the introductory slides to DZone. Photos of the event can be found here.

Dzone






View more presentations from Kathiravelu Pradeeban.

by Kathiravelu Pradeeban at April 25, 2012 07:46 PM


Kathiravelu Pradeeban—PART-2

The sequels

I had a series of blog posts that I always wanted to write a sequel for. Some of them were from this blog itself, and some are from the other authors, which I wanted to extend by writing the part-2 of them. All my posts depict my view as of that particular moment. A sequel may be an improvement on the understanding on the same, which can even be in a different view from its predecessor.



Second

However it didn't take much time for me to understand that extending is much difficult than writing something new. It is even harder, given that I don't like the idea to be second-hand. I personally hate to be the second in what I am really interested in doing. 





Public knowledge is public

Public knowledge should be open. Hate it when I  see, "To read the full text of this article and others like it, try us  out for 7 days, FREE". To make it worse, they become the first hit in the relevant google search. All the crappy search engine optimization (SEO) efforts of those guys!





On my way home

"On my way home" was initially the name of this blog. The thoughts during a long bus journey was always a major contribution or induction to write a new blog post, hence fitting the name of the blog. I changed it lately to "Llovizna", to fit the dynamic nature of the blog. "Llovizna" is "drizzle" in Spanish, also the name of a waterfall in Venezuela. Llovizna has seen a lot of blog posts. Some posts have a defined title, while some of them just summarize the random events in a form of a digest. This fits the latter, fitting as a sequel for multiple posts. Will try to expand these into separate blog posts. 2012 is going crazy and interesting. Hope to blog more later. :)




[Contd: Web History, The Planet, Plagiarism, WWW, Digital Marketing, Distributed Computing, shared, ..]

by Kathiravelu Pradeeban at April 25, 2012 07:38 PM


Kathiravelu Pradeeban—Incubators!

Incubators

The school or the 
university days are like living in an  incubator. Then comes the real 
world, where 1000x people are there to  confuse than motivate.
 Being independent helps. In times, we have received the flood of 
praises, and sometimes, the situation is reversed. While taking the 
message, we should keep moving on, without reacting much. 
Self-motivation is always the key. I can list one more - keeping the expectations
 low from others, while having the highest expectations from one's own 
self. (I will be satisfied, if I do, what I expect myself to be doing.)



To love or to judge

The
 world is full of judges and critics. It is not very difficult to find 
someone who can judge you, but it may be extremely difficult to have 
someone who loves without judging. Unlike in exams, we are evaluated 
each minute in the real world. Either you be smiling, or faking a smile
 to not appear someone who whines always. Labeling someone as an addict,
 pervert, or stupid is not effective in any way. Nevertheless, we can 
actually point the specific incident and help him do the corrective 
measures to overcome the addiction or problem. No one is the same 
always. 



3 

I
 have read somewhere - that in a cooperate world, the minimum overall 
rank an employee is given in a review is always 3 (meets expectations), 
in a 1 - 5 scale. This is to avoid demotiving the employees with the 
lower ratings. If you are rated 3 in an organization that adheres to 
those suggestions, be informed that you are ranked the least. :-)





Effort

Nevertheless, an effort is never wasted. 
Though the immediate outcome may not be positive, an honest effort on 
something always provides a long term gain. 

by Kathiravelu Pradeeban at April 25, 2012 07:37 PM


Madhuka Udantha—Clean Your Code with Sonar
What is Sonar?
  Sonar is an open source software quality platform. 
  Sonar use as static code analysis tool, which then can be used to improve software quality.
  How can we used Sonar for Improve our code quality?
     Download Sonar Server from here http://www.sonarsource.org/downloads/ 
    Unzip the ‘sonar-3.0.zip’
    Now start the sonar server in ‘\sonar-3.0\bin\windows-x86-64\StartSonar.bat’ (pick the correct sonar server depending on your running OS)
 
  
  wait some moment till the server start (It will take around 30 seconds)
  4.  Then in the project directory enter ‘mvn sonar:sonar’ (For the first time this take some times depend on you project size) 
  [Note] ‘mvn sonar:sonar –o’ is for offline 
  
  5.  Installing Sonar and PMD plugins for Eclipse (Here is the post how to install sonar plugin for eclipse)
  6.  In Eclipse go to the project and right click on it. Configure – > Associate with sonar   
  
  7.  Check your project in the list. Then click on the ‘find on server’ button. Finally click on ‘Finish’ button 
  
  8.  In Eclipse go to ‘Sonar perspective’ view (button is placed in  right hand side upper corner)
  
  9.  In below violation tab will show all the violation in  the code with description. just click on that particular code snippet will be open. (Check other tabs also) 
  
  [Note] code  changes will be show after we run mvn sona: sonar 
  10.  Right click on the project “Sonar ---> Open in sonar server”  open in web browser “localhost:9000”
  
  
  Here is the code review from Sonar 
  
by Madhuka at April 25, 2012 12:50 PM


Ganesh Prasad—Dimensions of Decoupling

I was in a meeting at work discussing deployment strategies for various SOA components. Let me take a subset of the problem as today's topic.

The issue was how to deploy a bunch of ESB instances and App Server instances onto server boxes. One of the infrastructure guys said he preferred to have one ESB instance and one App Server instance on each box for ease of administration. Now, since we were going to run the instances on virtual servers, I suggested that we not worry about it at a SOA topology level. We would only talk about virtual servers running ESB instances and virtual servers running App Server instances. If the infrastructure guy wanted, he could always run a virtual server of each type on each physical box and would then get what he wanted. He had the ability to tune the configuration, allocating 0.5 CPUs to each virtual server, etc.

He objected to that idea, saying the performance of the virtual servers was bad if he deployed them like that. He wanted to deploy one instance of the ESB and one instance of the App Server on each virtual server.

I said, "OK, then deploy the two instances on the same virtual server, but don't assume that they're on the same virtual server."

This statement was a bit too cryptic for the others in the room, and they asked me to explain. This is how I explained it:

Assume that you have to write a deployment script that installs one instance of the ESB and one instance of the App Server on a virtual server. There are at least two ways you could do it.

Script option 1:
DEPLOY_ADDR=192.168.1.2
# Deploy ESB to $DEPLOY_ADDR
# Deploy App Server to $DEPLOY_ADDR

Script option 2:
ESB_DEPLOY_ADDR=192.168.1.2
AS_DEPLOY_ADDR=192.168.1.2
# Deploy ESB to $ESB_DEPLOY_ADDR
# Deploy App Server to $AS_DEPLOY_ADDR

In both cases, the scripts will deploy one instance of the ESB and one instance of the App Server onto a single server.

However, in the first script, the two servers are assumed to be the same. In the second, they happen to be the same. That is the difference between tight coupling and loose coupling.

In the second script, a simple change in the value of (say) AS_DEPLOY_ADDR to 192.168.1.3 will see the two instances running on different servers. This is not possible with the first script. Changing DEPLOY_ADDR to 192.168.1.3 will move *both* instances to the new server address but will not separate the two instances.

I was therefore recommending the approach exemplified by the second script - deploy the two instances on the same server if you must, but don't hardcode the assumption that they are on the same server into your scripts.

It took a while for this concept to sink in, but the idea was finally accepted.

I guess a decade of SOA experience has sensitised me to looking out for needless dependencies, but most people in IT still don't think this way. I wonder how much rigidity and subsequent operational inefficiency is in IT systems all over the world because people are not sensitised towards the elimination of needless dependencies.

by Ganesh Prasad at April 25, 2012 08:36 AM

April 24, 2012


Sanjeewa Malalgoda—Carbon Context and it's usages in WSO2 Products
A context can be defined as a construct that strictly bounds the execution environment. Because Carbon Context is a runtime container for your app(s) you gain the benefit of leveraging the Carbon Context runtime API allows to obtain contextual information about various actors utilizing your web apps and web services. For example, you may want to use registry,cache for that particular user or tenant. The carbon context allows you to access 5 APIs and has several utility methods. In each time we create user specific thing we create carbon context for that user to store specific things. This is valid for all WSO2 Carbon 3.2.2 based servers.

Here is the link to post written by regarding carbon context and its usages.
http://wso2.org/library/articles/2012/03/carbon-context-its-usages-wso2-products

by Sanjeewa Malalgoda at April 24, 2012 05:25 AM

April 23, 2012


Srinath Perera—Generating a Distributed Sequence Number
This is a very common problem in distributed systems (e.g. Message brokers, implementing "At most once deliver", Group communication etc). I was doing some reading for WSO2 Andes project. 
There are several options.
Using Zookeeper: Following two threads talk about this. It should be reasonably fast. Twitter guys have tried this and says it was bit slow.
http://zookeeper-user.578899.n2.nabble.com/Sequence-Number-Generation-With-Zookeeper-td5378618.html
http://www.mail-archive.com/zookeeper-user@hadoop.apache.org/msg01976.html
Cassandra:  This has been raised several times, and answer was to use UUIDs (which does not work for us)
http://comments.gmane.org/gmane.comp.db.cassandra.user/3304
 http://stackoverflow.com/questions/3935915/how-to-create-auto-increment-ids-in-cassandra.

 Then Cassandra introduced counters,  but it does not support incrementAndGet() and no plan to do the future as well.   So that does not work. 
http://www.datastax.com/dev/blog/whats-new-in-cassandra-0-8-part-2-counters
Write a custom server:  This is easy, basically create a service that give a increasing ID. But very hard to cluster this and behavior in case of a failure is complicated. 
"A timestamp, worker number and sequence number":  Twitter Guys created solution based on "a timestamp, worker number and sequence number" (this is kind of that we use as well, except that ran few dedicated servers for this)   http://engineering.twitter.com/2010/06/announcing-snowflake.html
Other Algos: Only looked at these briefly. But they are complicated. 
 Using DHTs:        http://horicky.blogspot.com/2007/11/distributed-uuid-generation.html
A Fault-Tolerant Protocol for Generating Sequence  Numbers for Total Ordering Group Communication  in Distributed System, http://www.iis.sinica.edu.tw/page/jise/2006/200609_16.pdf
IMHO, "a timestamp, worker number and sequence number" is the best option. Only downside of this is that this assumes that broker nodes are loosely synced in time.   Only other option I see is Zookeeper.

Good overview - http://stackoverflow.com/questions/2671858/distributed-sequence-number-generation/5685869  

by Srinath Perera at April 23, 2012 06:33 AM

April 22, 2012


Denis Weerasiri—Introduction and Advanced Concepts of BPEL
Waruna and I held a tutorial session for WSO2Con-2011 post conference tutorials. We basically explained and demonstrated on what the WS-BPEL constructs are and how to write a simple WS-BPEL process. And also we discussed some advanced concepts related to WS-BPEL as well. Here I have attached the slide-deck we presented. 
Introduction and Advanced Concepts of BPEL  
View more presentations from Denis Weerasiri 

Hm... it's been quite a messed-up week after all. Tomorrow I'm going for some snorkeling+diving with some of my friends :). Have a nice week-end!.

Related post: http://ddweerasiri.blogspot.com/2011/08/advanced-concepts-in-ws-bpel-20.html

by Denis Weerasiri at April 22, 2012 06:33 PM


Denis Weerasiri—Business Process Hosting in the Cloud
I wrote an OT article named "Business Process Hosting in the Cloud". This article discusses multi-tenancy aspect in a cloud computing environment and some of the concerns of hosting business processes in the cloud. WSO2 Business Process as a Service supports development and easy deployment of business processes modeled using the WS-BPEL standard in a multi-tenanted environment. Also it provides functionality for business process management and hosting in the cloud.

by Denis Weerasiri at April 22, 2012 06:32 PM


Denis Weerasiri—Enabling logging for various components in WSO2 BPS
This blog post explains how to log the messages come into and sent out from WSO2 BPS server.
This feature is extensively used in BPEL development to figure out the problems in the message sequences and in latency analysis during BPEL process invocations.

Tested Environment
     wso2bps-2.1.2

Instructions
Add the following entries to the $CARBON-HOME/lib/log4j.properties
    log4j.logger.org.apache.ode.bpel.messagetrace=TRACE
    log4j.logger.org.wso2.carbon.bpel.messagetrace=TRACE
Then the preferred log4j appender should be configured such that it has a threshold of TRACE level. If CARBON_LOGFILE is the log4j appender, it should be changed as follows. By default this is set to DEBUG.
    eg - log4j.appender.CARBON_LOGFILE.threshold=TRACE
Then re-start the WSO2 BPS server.
The log files can be found as $CARBON-HOME/repository/logs/wso2carbon.log 
Note - You can configure this via Management console as well.

by Denis Weerasiri at April 22, 2012 06:32 PM

April 21, 2012


Sagara Gunathunga—Axis2 JMS transport and ActiveMQ

In this post I will describe how to configure Axis2 JMS transport 
properly and test web services through JMS transport. For the simplicity
 I will use Apache ActiveMQ as the JMS server and will use ActiveMQ 
admin console as a JMS client to send and receive messages. In my future
 posts I will describe how to use ActiveMQ Maven plug-in with Axis2 and 
also how to test Axis2 JMS transports with few other implementations 
such as Apache QPID and WSO2 Message Broker.



First, If you don't
 have a ActiveMQ installation already, download the binary distribution 
from here and start the ActiveMQ server. If it's started properly make 
sure you can access to admin console through the following URL, we will 
use this admin console as a JMS client.

 http://0.0.0.0:8161/admin  

The next step is add requited dependencies and configure JMS transport 
in Axis2. Here I use Axis2 Simple HTTP server but same steps can be used
 with any other application server too.



(1) Add following dependencies to the "lib" directory of Axis2.



1.  axis2-transport-jms-1.x.x  (axis2-transport-jms-1.7.0-SNAPSHOT.jar or axis2-transport-jms-1.7.0 )

2.  axis2-transport-base-1.x.x (axis2-transport-base-1.7.0-SNAPSHOT.jar or axis2-transport-base- 1.7.0)   

3. geronimo-j2ee-management_1.1_spec-1.0.x  (geronimo-j2ee-management_1.1_spec-1.0.1.jar)

4. geronimo-jms_1.1_spec-1.1.x  (geronimo-jms_1.1_spec-1.1.1.jar)

5. activemq-core-5.1.x (activemq-core-5.1.0.jar)

6. coomons-io-2.1 (coomons-io-2.1.jar)



You can find above dependencies on Axis2 transport project here or you 
can find latest development snapshots from Apache build server here.



(2) Like in any other Axis2 transport your next task is to configure 
particular transport through the axis2.xml file by adding underline 
TransportListener  and  TransportSender. For the JMS transport you can 
add following settings.



 <transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener">  
   <parameter name="default" locked="false">             
     <parameter name="java.naming.factory.initial" locked="false">org.apache.activemq.jndi.ActiveMQInitialContextFactory</parameter>  
     <parameter name="java.naming.provider.url" locked="false">tcp://localhost:61616</parameter>      
     <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">QueueConnectionFactory</parameter>  
   </parameter>  
 </transportReceiver>  



 <transportSender name="jms" class="org.apache.axis2.transport.jms.JMSSender">  
   <parameter name="default" locked="false">             
     <parameter name="java.naming.factory.initial" locked="false">org.apache.activemq.jndi.ActiveMQInitialContextFactory</parameter>  
     <parameter name="java.naming.provider.url" locked="false">tcp://localhost:61616</parameter>      
     <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">QueueConnectionFactory</parameter>  
   </parameter>  
 </transportSender>  




Axis2 Transport user guide provide advanced configuration details such as separate configuration for JMS Queue and Topic etc.



(3) Start Axis2 server, If JMS transport is configured properly it is possible to see log message about JMS transport as follows.


 [INFO] JMS Sender started  
 [INFO] JMS ConnectionFactory : default initialized  
 [INFO] JMS Transport Sender initialized...  
 ..................  
 [INFO] JMS ConnectionFactory : default initialized  
 [INFO] JMS Transport Receiver/Listener initialized...  
 [INFO] Listening on port 8080  
 [INFO] JMS listener started  
 [INFO] Task manager for service : Version [re-]initialized  
 [INFO] Started to listen on destination : Version of type Queue for service Version  
 [INFO] Task manager for service : CalculatorService [re-]initialized  
 [INFO] Started to listen on destination : CalculatorService of type Queue for service CalculatorService  
 [INFO] Task manager for service : mtomService1Axis [re-]initialized  
 [INFO] Started to listen on destination : mtomService1Axis of type Queue for service mtomService1Axis  
 [INFO] [SimpleAxisServer] Started  
 [SimpleAxisServer] Started  



Now you have configured Axis2 JMS transport properly and it's possible 
to use any JMS client to invoke web services deployed on Axis2.  Let's 
try to invoke getVersion operation on Version web service through 
ActiveMQ admin console. Again in a browser go to the ActiveMQ admin 
console through the http://0.0.0.0:8161/admin URL, now  you have to 
select "Send" tab to reach the send wizard.  This wizard expect 3 basic 
inputs from you as input message, JMS destination ( in this case 
destination queue) and reply-to location. By looking at Axis2 server 
startup logs you can find hint about JMS destination bind to each web 
service, you can ensure availability of this JMS destination in ActiveMQ
 console too.

  [INFO] Started to listen on destination : Version of type Queue for service Version  





For our test scenario let's use following properties.


 Destination - Version
 Reply-To    - VersionResponse ( This destination is not available on 
ActiveMQ at this point, it will be created when the repose messages arrive to 
the JMS server)
In put message - Use following message
Correlation ID 
 - Add a random number

 <soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:axis="http://axisversion.sample">  
   <soap:Header/>  
   <soap:Body>  
    <axis:getVersion/>  
   </soap:Body>  
 </soap:Envelope>  









Now you can see the response message by browsing "VersionResponse" Queue as follows.










by Sagara Gunathunga at April 21, 2012 03:35 PM

April 19, 2012


Denis Weerasiri—WS-BPEL 2.0 Extension Activity Development for WSO2 BPS
There are several ways to extend WSO2 BPS runtime functionalities such as
BPEL extension activities
Custom XPath extensions
Message Exchange Interceptors etc.

BPEL extension activities enable a pluggable architecture that allows for registering third party functionality to the WS-BPEL 2.0 execution runtime.
There are several BPEL extension activities supported by WSO2 BPS and with the upcoming WSO2 BPS 3.0.0 release, we are going to introduce several new BPEL extension activities such as
BPEL4People Activity - Enables human interactions based on WS-HumanTask specification via BPEL processes using BPEL4People specification.
Business Activity Monitor Activity - Enhances real time KPI(Key Performance Indicators) monitoring via WSO2 BAM

Here I have written a simplest extension activity implementation with a sample BPEL process so, a BPEL developer can re-use and extend it to have their own implementation.
The extension activity implementation can be found from https://svn.wso2.org/repos/wso2/people/denis/org.wso2.carbon.bpel.extension.activity.sample/.

To come up this implementation, I followed this blog-post from Waruna Ranasinghe and it brings you more in-depth knowledge on BPEL extension activity development.

by Denis Weerasiri at April 19, 2012 11:33 AM


Madhuka Udantha—Installing Sonar and PMD plugins for Eclipse
If you are new to Eclipse IDE, Better to know how to install Plugin to it. This Post will be explained “how install Plugin to Eclipse”. 
  1. Start Eclipse IDE ( Screen shoot from the Eclipse Java EE IDE for Web Developers, version: Helios Service Release 2)
     
2. Select the Help –>Install New Software.... in eclipse menu item.
  
     
3) Install window will be open in there click “Add..” button.
       
  4) In “Add Repository” window enter the Name and Location (plugin URL)
  PMD plugin: http://pmd.sf.net/eclipse
  Sonar plugin: http://dist.sonar-ide.codehaus.org/eclipse/
  Then Click “OK” button.
       
 5) Select PMD for Eclipse 3 and Apache Xerces in the "Select the features to install" list and click Next.   
Accept the terms of the license agreements and click Next.   
  
  6) Verify that the install location in Eclipse installation directory, otherwise select the correct one, click Finish.    
  [Note] For the Sonar 
  
  
by Madhuka at April 19, 2012 06:09 AM

April 18, 2012


Chris Haddad—How to pick an ESB? An Enterprise Service Bus Evaluation Framework
All Enterprise Service Bus (ESB) products may be used to build and deploy services, encapsulate legacy systems, route messages, transform message formats, and perform protocol mediation.  Many WSO2 prospects ask me ‘What differentiates WSO2 Enterprise Service Bus?’  This blog post shares my perspective and scales the conversation.

Integration teams use ESBs to solve service-oriented anti-patterns of isolation, uniqueness, and duplication.  Isolation refers to stand-alone system silos and point-to-point connections between systems (as opposed to shared connections).  Unique data schema models for similar entities, transactions, and processes raise integration cost.  When an organization fields multiple software applications delivering similar business functions and requires redundant data entry, opportunities exist to remove duplication and save both operational expense and maintenance cost.
To reduce service-oriented anti-patterns, teams desire to share and re-use assets, consolidate functionality, and conform projects to common standards.   To achieve these best practices, teams rely on an ESB to deliver the following required architectural attributes:

Interoperability
Abstraction
Resource location virtualization
Ability to scale and manage service
Declarative policies and platform independent models
Separation of concern
Loose coupling

Architectural attributes are hard to measure, and we find most evaluation teams do not develop evaluation use cases across all architectural attributes.  Teams often focus on easy identifiable product features.  ESB products may contain the following required and optional features:
Required features

Routing
Protocol bridging
Message transformation
Service agent hosting

Optional features

Resource adapters
Composition
Orchestration
Reliable message delivery
Event processing
Transactional integrity
Message Exchange Pattern (MEP) mediation
Dynamic location and binding, load balancing
Message validation
Capability mediation
Security mediation (federation)
Tooling

After creating use cases testing an ESB’s ability to deliver architectural attributes and features, an ESB evaluation process should also consider how the ESB fits into the complete platform.  An ESB is usually just a single component in a broader composite SOA Platform.   Teams often combine an ESB with Governance Registries, Identity Servers, Business Activity Monitoring, Complex Event Processing, and Business Process Management.  Teams often differentiate ESBs based on how well the ESB fits into a complete solution.  A platform built by vendor innovation instead of vendor acquisition will often provide a more holistic and cohesive experience by sharing meta-data, administration consoles, programming models, and foundational features (i.e. logging, security, management, provisioning).
Performance, scalability, and topology strongly influence solution agility and adaptability.  We see organizations evaluating how well ESB candidate products fit across hybrid Cloud environments (i.e. on-premise, outsourced, internally managed, externally managed [1]), high transaction loads, and low latency use cases.  Teams should devote ample time to build suitable performance, scalability, and topology tests.
A Proof of Concept project provides an opportunity to ‘kick the tires’ and evaluate the ESB’s ability to satisfy use cases.   Rather than simply relying on vendor demos, download the bits and directly experience the ESB’s programming model, administration interfaces, documentation, and samples.   During the Proof of Concept project, carefully evaluate the vendor’s support processes, openness [2], responsiveness, and ability to recommend suitable solutions.
By carefully considering your requirements, constraints, and technology strategy, your team can build an ESB evaluation framework demonstrating product similarities and strategic differences.  A comprehensive, weighted evaluation criteria set will ensure the ESB meets your needs today and in the future.  An evaluation framework mindmap is shown below:
Enterprise Service Bus Evaluation Framework
 
[1] http://blog.cobia.net/cobiacomm/2011/11/07/know-your-cloud-dimensions/
[2] http://blog.cobia.net/cobiacomm/2012/03/14/value-openness/
 
by haddadcblg at April 18, 2012 08:39 PM


Hasini Gunasinghe—Notes from IETF 83rd Meeting
As you may know, IETF 83rd meeting was held in Palais des Congres, Paris from 25th-30th March. I too got the opportunity to attend the IETF 83rd Meeting and SCIM Interop Event which was held in parallel to it.

It was an interesting, novel and great experience to see how people who form technology standards-(that we implement and that become buzzwords in the industry), get together in IETF meetings as working groups (WG) and present new ideas, discuss and argue on them, conduct consensus and agree upon things which is a part of the whole long process of publishing a standard as a IETF RFC.

New comers  orientation:

OK, first let me mention what is IETF, its purpose and how it operates as I learned from this session which was held on Sunday 25th March.  Scott Bradner - Secretary of Internet Society explained the the what & the how of IETF to all new comers. 
IETF (Internet Engineering Task Force) is the organization that develops and maintains the standards related to how the internet operates today. It meets 3 times a year.
It is an open organization that any one can join through mailing lists and contribute to the development of standards of your interested area.
There are 8 main areas of focus: Application, General, Internet, Operations & Management, Real-time Applications & Infrastructure, Routing, Security, Transport.
There are 131 Working Groups under the above areas - it is in a working group that the standards are developed. Each working group has a mailing list where the work happens.
IETF Management consists of : IETF Chair,Area Directors (AD), Internet Engineering Steering Group (IESG), Internet Architecture Board (IAB).
IETF management are all volunteers. People are company or self supported.
RFC are the final document published by IETF. Although earlier it referred to as 'Request for Comments', no changes made after RFC is published. So now RFC is not an acronym.
It usually takes about 2 - 3 years for a draft-00 version of a technical standard be published as a RFC.

Tutorial sessions for beginners:

First day(Sunday afternoon) was allocated for induction & tutorial sessions. 
- I attended one tutorial session on "Operations, Administration, and Maintenance Tutorial" which was focused on networking side. 
- There was another tutorial on "Tools for Creating Internet-Drafts Tutorial" which I think would have been more useful, but  I missed it since the session was held in parallel to orientation session mentioned above.
- Slides of both these sessions can be found here under Training.
 

Above are two pics of Palais Des Congres where IETF 83rd meeting was held.

Meetings.. Meetings.. Meetings..

Mainstream IETF work started from Monday onwards and there were meetings of several types throughout the week as I have categorized below:
- Birds of feathers sessions : these sessions are conducted to decide whether a working group should be formed inside IETF to carry on work of a new standard. Only very few sessions of this type are held in one IETF meeting.
- WG meetings : Majority of meetings fall under this category. This is where WG members meet and discuss about the issues in current drafts formed by that particular WG and present the new drafts to IETF etc.
- Informal meetings organized by other societies/communities : Related organizations and communities like Internet Society and WGs from OASIS etc. take the free slots in IETF agenda (like lunch break) and conduct sessions on the topics of current interest. These are informed through IETF registrants' mailing list and participation is allowed through first come fist served basis.
- Technical plenary sessions : All most all IETF attendees who attended to different WG meetings of their interests, attend these plenary sessions where reports of different IETF management groups (like IAB, IRTF) are presented and a technical topic of common interest is discussed.
- Research forums: 
These are conducted by research groups chartered under IRTF (Internet Research Task Force) which  is an affiliated organization of IETF. It focuses on long term research problems related to internet.

I happened to attend all types of meetings during the week. Several sessions happen in parallel and sometimes we miss some of the interesting sessions as well.
SCIM BoF - Simple Cloud Identity Management is an emerging standard for user account and identity provisioning. It was proposed to be chartered under Application Area of IETF as a working group. BoF session went full house even before the session starts. Morteza and Trey explained "the what" & "the how" of SCIM to the IETF community. The session was chaired by two Area Directors.
Security is a key aspect discussed in all the above types of IETF meetings and and there is a separate Area (out of 8 focus areas mentioned above) dedicated to security.

Since my area of focus at WSO2 and also my personal interest and passion lies in the area of security, I decided to attend the Working Group meetings under Security focus area of IETF which I have listed below.
You can find the slides of these sessions here under Security Area.
Web Security WG
Public Key Infrastructure
Kerberos WG + KITTEN
Java Script Object Signing
OAuth
Security Area Open Meeting
It is interesting to see the process of how the security related standards that we implement and use are being formed at IETF.

 

Above are pics of IETF crew during the tea break... 

I also attended two informal meetings organized by other organizations/communities.
Authentication and Authorization: Next steps for OpenID and OAuth: this was organized by Internet Society Trust & Identity Initiatives. The panel discussed about OAuth, building security tokens based on JSON data/JWT, OpenID, ID token, adding identity layer to OAuth & Web cryptography working group. You may find the full audio of the session at the above link.
 Beyond HTTP Authentication: OAuth, OpenID, and BrowserID: this session was hosted by W3C and discussed about OpenID Connect, Key differences from OpenID 2.0, Browser ID, use cases of OAuth + Browser ID.
Out of the meetings from IRTF, I attended to:
Crypto Forum Research Group : this was the last session I attended in IETF meetings. You can find the slides here under IRTF -> CFRG. 
Out of the two plenary sessions, I attended the Technical Plenary session which discussed about Implementation Challenges for Browser Security which was a very fruitful discussion with lot of involvement from audience as well.
You may find the slides of this session here under Plenary Sessions -> Technical Plenary.


Above is a pic taken during the technical plenary session...

Remarks: 
- There were many people involving in developping standards - not only from Universities, but also from different companies.
- There were only 2 other Sri Lankans - one from a German University and the other from Cisco. 
- I believe it will be great if there is more involvement/impact from Sri Lankan Universities/Companies also.
- It was a very valuable opportunity have participated in IETF meeting and witness how the standards that we implement are actually formed and which in fact was a great community meet up as well.

by Hasini at April 18, 2012 08:06 PM

April 16, 2012


Chris Haddad—What is WSO2 AppFactory?
Application development organizations continue to undergo a structural shift towards business enablement and away from technical debt. Teams desire to re-invent software delivery into an agile, on-demand application environment and change the business-IT dynamic. WSO2 platforms enable IT to solve mundane technical plumbing and focus on business-oriented personalization, self-service, monetization, and analytics.  When changing the business-IT dynamic, we see leading clients:

Share infrastructure and improve internal software delivery
Enable on-demand digital disruption via ecosystem platforms

Share infrastructure and improve internal software delivery
Development teams desire to increase efficiency by sharing infrastructure and improving software delivery.  WSO2 Stratos delivers seventeen distinct on-demand middleware services.  Cloud tenants may subscribe to WSO2 Stratos’ multi-tenant shared services and eliminate traditional application platform provisioning time lag.
Teams desire to enhance agile development methodologies by integrating DevOps practices and on-demand application life-cycle infrastructure. Teams often are burdened by a need to provision a complex set of project tracking, source code management, issue tracking, test tools, release management scripts, and run-time application platform environments (i.e. development, test, production).  WSO2 AppFactory applies on-demand self-service and automated provisioning patterns to software projects.  The WSO2 AppFactory environment delivers multi-project software delivery infrastructure tools and encourages software delivery best practices. WSO2 AppFactory enable teams to create and administer projects, automatically build applications, provision quality assurance environments, run continuous integration tests, and continuously promote and deploy software assets across development life-cycle environments (i.e. Development, Test, and Production).  Figure 1 provides a visual illustration.
WSO2 AppFactory
Enable on-demand digital disruption via ecosystem platform
Technology consumerization, capability externalization, and content creation/sharing democratization are disrupting traditional business models [1,2].  Startups and upstarts are rapidly disinter-mediating and replacing brand-name business providers, brokers, and distributors.  Business leaders are realizing new digital strategies are required to build relevance and connections across their suppliers, customers, and employees.
A new application platform environment is required to increase IT interaction efficiency, deliver composite business service offerings, and enable new revenue sharing opportunities.  Our clients augment WSO2’s Cloud Native application platform, WSO2 Stratos, with shared business capabilities, custom showback/chargeback models, and application hosting to create an AppFactory delivering digital disruption to internal IT delivery. We have productized the digital disruption and ecosystem patterns in our WSO2 AppFactory offering.
An ecosystem platform helps organizations extend their core business capabilities across application projects.  A multi-tenant, extensible cloud environment is used to personalize channel interactions and deeply embed business capabilities within a business partner’s distribution channel (similar to Force.com, eBay sellers, or Amazon Store environments).  By natively including on-demand self-service and automated provisioning, the platform enables efficient engagement with ad hoc departmental and line of business projects (i.e. the long tail) and ability to quickly scale.  By hosting all application projects within a multi-tenant environment, the environment can easily aggregate and share business information.
 
 
[1] http://mashable.com/2011/10/27/digital-disruptors/
[2] http://socialmediatoday.com/david-h-deans/469985/sxsw-2012-digital-disruption-continuum
by haddadcblg at April 16, 2012 06:45 PM

April 13, 2012


Denis Weerasiri—Writing a BPEL compensation handler
Compensation handling plays an important role in Business Process Management. It specifies how to undo a particular task which is not completed due to some error. Let’s take an example.
eg - Take a simple travel booking process which includes three parallel sub-tasks as follows.
Travel Booking Process 

Now let’s assume hotel and taxi booking were successful. But couldn't book the flight. Now the “Book Hotel” and “Book Taxi” sub tasks should be compensated in order to mitigate the loss.

In WS-BPEL 2.0, compensation handling is specified using <compensationHandler/>. And <compensateScope/> or <compensate/> can be used to trigger the <compensationHandler/>.

Let’s take a look at the how a compensation handler can be implemented.
The easiest way is to add <compensationHandler/> in-line with <invoke/> activity. So when a fault occur; the in-line <compensationHandler/> is triggered.  
eg - 

The other method enables to compensate a specific scope using <compensationHandler/>. Here the <faultHandlers/> in the parent scope is used to trigger the <compensationHandler/> in the immediate child scope. For detailed descriptions on how exactly to trigger a <compensationHandler/> refer “12.4.3. Invoking a Compensation Handler” in WS-BPEL 2.0 specification.
eg - 

We have added a sample compensation handler which is an implementation of a simple “Flight reservation process” to WSO2 BPS BPEL-2.0 sample repository.

by Denis Weerasiri at April 13, 2012 05:53 PM


Denis Weerasiri—Wishing you a very happy new year 2012!
Since it is the festive season of Sinhala and Hindu new year for all Sri Lankans, I wish you all a happy and prosperous New Year 2012!. Also my team is celebrating the new year dawn with their loved ones except for few guys at on-site work outside LK.

With just the beginning of the vacation, we spent few days at Heritence Ahungalle, a beach hotel on the west coast of Sri Lanka. I had a luxury suite thanks to a good old friend from school days :). But we spent most of the time in the swimming pool and the beach due to the hot and humid weather.
Also the hotel is quite renowned for its sustainable building architecture by Geoffrey Bawa.


Related Posts
http://ddweerasiri.blogspot.com/2012/01/2012-has-arrived.html
http://ddweerasiri.blogspot.com/2011/04/wishing-you-very-happy-new-year.html
http://ddweerasiri.blogspot.com/2010/02/happy-new-year-sri-lanka.html


by Denis Weerasiri at April 13, 2012 02:03 PM


Sagara Gunathunga—How to use Axis2 WSDL2JAVA with proxy authentication
Securing WSDL using basic authentication is a common practice for number  of enterprise web service providers. Also in some business domains it's  required to secure various WSDL URLs among business partners so that  partners can access service contracts according to their business  agreements by providing a user name and a password.  As the most popular  Java web service stack lot of people keep asking to enable basic   authentication to WSDL2JAVA tool so that they can access WSDL files  secured through basic authentication.



Now you can use following two options to specify proxy user name and password.


 -http-proxy-user [user name]      - Proxy user name for basic authentication.  
 -http-proxy-password [password]   - Proxy password for basic authentication.  


The following examples show how to use above option in command line.



 wsdl2java.sh -http-proxy-user user-name -http-proxy-password password -uri http://localhost/axis2/services/Version?wsdl  


 wsdl2java.sh -http-proxy-user user-name -http-proxy-password password -wv 2.0 -uri http://localhost/axis2/services/Version?wsdl2   


Also these two options supported in Wsdl2Java Maven plug-in too.



 axis2.wsdl2code.http-proxy-user  - User name for proxy server.  
 axis2.wsdl2code.http-proxy-password - Password for proxy server.   








by Sagara Gunathunga at April 13, 2012 01:37 PM


Denis Weerasiri—Development and Deployment Best Practices for WSO2 BPS
This post contains an updated version of best practices related to developing and deploying business processes on WSO2 BPS (Business Process Server).

Table of Content
Deployment best practices
Development best practices
Deployment best practices
Default distribution of WSO2 BPS comes with embedded H2 database as BPEL engine's persistence storage and other settings which are suitable for use in development environment. But when you are going to production with WSO2 BPS, there are several configurations you need to change according to your production requirements. These configurations will change based on how much requests BPS is going to handle per second, your auditing and monitoring requirements, performance requirements and nature of your process. Following are the main things you should do before going production with WSO2 BPS.
If the deployed BPEL processes has any conflict like
    - Same BPEL process is deployed under several different package names
    - When a deployed BPEL process has an existing service name
Those deployment issues are displayed under the relevant package in "Deployed Packages" page.
Configure external database server like MySQL as your persistence storage instead of embedded H2 database. You may experience slight performance gain for simple BPEL processes with H2 database, but when it comes to multiple concurrent requests and complex processes H2 can't server your performance needs.
Configure multi-threaded Http connection manager connection pool settings to suits to your BPEL processes. There are two configurations in Http connection manager. One is max total connections and other is max total connection per host. These settings will depend on number of concurrent requests BPS needs to handle and number of external service calls incorporated per process instance.
Configure BPEL process persistence - If you are implementing processes with request-response interaction model use in-memory processes instead of persistence processes. Whether to use in-memory or persisted processes will mainly depends on your business use-case.
Configure even-filtering at process and scope level. So you can save lot of database resources by reducing number of events generated.
Use process-to-process communication, if you are calling one BPEL process from another BPEL process deployed in the same BPS instance, it's better to use process-to-process communication to reduce overhead introduce by additional network calls.
In the default WSO2 BPS distribution, the size of a fault message (which is stored in BPEL DB) is limited to ~4KB.

eg - See the following BPEL database SQL script
So if a deployed BPEL process is expected to handle larger size of fault messages, the above database script should be modified and re-built from the source distribution accordingly.
Also make sure to configure process instance cleanup. Large number of process instance data will be accumulated in the BPEL engine persistence storage if you persisted processes, so to reduce performance overhead introduced by database size you should configure instance cleanup.
In addition to above things you should be careful when deploying WSO2 BPS in virtualized environments. We have seen random increase of network latency and random performance degradation when running on VMs.
If the BPEL is going to be deployed as a WS-secured service then, it's recommended to remove all the http endpoints from the process WSDL. Else the WSDL generation for the particular BPEL process will get failed.

eg -
SHOULD BE
Note 1: Above mention configuration optimizations are true for Apache ODE also.

Note 2: Above mention best practices are valid for WSO2 BPS 3.0.0-SNAPSHOT and upward. You can do the above optimizations to older versions WSO2 BPS, but configurations and configuration mechanisms will be different. All of the above optimizations are supported by Apache ODE, but configuration is very different from WSO2 BPS.

Development best practices
When it comes to BPEL development in WSO2 BPS, BPEL developer need to aware some scenarios which could lead to some conflictions. Those are listed as follows.
It's not encouraged to refer the same variable as the input(in <receive/>) and output(in <reply/>) of the process. This could lead to problems if the message headers (<Header/> in SOAP <Envelope/>) in output variable are processed at the client-end. One possible use-case is when the process is secured if the input and output variables are same then the headers of the input will be used when the output is sent back to the client. So it could prone to errors if those security headers are not expected at the client end.

by Denis Weerasiri at April 13, 2012 01:01 PM

April 10, 2012


Hiranya Jayathilaka—WSO2 ESB Tips & Tricks 09: Introduction to REST APIs
REST API support is one of the coolest features we have added to WSO2 ESB in the recent past. It was first released in last December along with WSO2 ESB 4.0.3, and we have already seen a fairly large number of users embracing the new feature. I blogged about this addition in last January, in an introductory post about the WSO2 ESB 4.0.3 release. 
Today I intend to briefly describe how we can start using this feature in simple integration scenarios. Assume you have an account management service implemented on a SOAP stack such as Apache Axis2. Your service may support basic crud operations such as create-account, read-account, update-account and delete-account. The create-account operation would take several inputs and create the account in a backend business system. Usually each account would be associated with some sort of a unique ID which can be passed in as a parameter to read, update and delete operations. Now assume that you want to expose this SOAP service through a RESTful API. Many architects and developers are currently looking for ways to easily expose existing SOAP services over REST, so that the services can be easily consumed by mobile clients and other web clients. With the REST API support in WSO2 ESB, such a REST to SOAP conversion is like an evening stroll in the park. Here’s the API configuration for the create-account operation.
<api name="AccountManager" context="/accounts">
 <resource methods="PUT">
    <inSequence>
       <payloadFactory>
          <!-- Create custom SOAP payload -->
       </payloadFactory>
       <send>
          <endpoint>
              <address uri="http://backend.service/url" format="soap11"/>
          </endpoint>
       </send>
    </inSequence>
    <outSequence>
       <property name="HTTP_SC" value="201" scope="axis2"/>
       <send/>
    </outSequence>
 </resource>
</api>
The above API can be invoked by sending a HTTP PUT request to the URL http://host:port/accounts. The payload of the request should have all the parameters required to create the account encoded as a XML document, a JSON string or any other suitable format. The ESB would do the REST to SOAP conversion and invoke the backend SOAP service. The response from the backend service will be sent back as a RESTful response with the HTTP status code 201. If possible it is also possible to add a custom Location header to the outgoing response. In a proper RESTful design, the Location header should be URL which points back to the read-account operation of the REST API.
Similarly more resource elements can be added to the above API to expose read, update and delete operations of the backend service. The full configuration would look something like this.
<api name="AccountManager" context="/accounts">
 <resource methods="PUT">
    ...
 </resource>
 <resource methods="GET" uri-template="/account/{accountId}">
    ...
 </resource>
 <resource methods="POST" uri-template="/account/{accountId}">
    ...
 </resource>
 <resource methods="DELETE" uri-template="/account/{accountId}">
    ...
 </resource>
</api>
Note how we map each HTTP verb to a unique operation in the backend service. Also note the usage of URI templates to specify request URL patterns to accept. WSO2 ESB has full support for level 1 and level 2 URI templates as described in the URI templates draft specification. Alternatively you can also use servlet style URL mappings to configure your API resources.
<api name="AccountManager" context="/accounts">
 <resource methods="GET" url-mapping="/account/*">
    ...
 </resource>
</api>
I believe the above example gives a basic idea on how the REST API support in WSO2 ESB can be used to implement real-life integration solutions. While this feature can have many potential use cases, one of its most powerful capabilities is to expose existing SOAP services and legacy systems over REST. Thanks to the REST API support in WSO2 ESB, implementing a RESTful overlay for an existing application is easier than never before.

by Hiranya Jayathilaka at April 10, 2012 02:52 PM


Charith Dhanushka Wickramarachchi—Distributed Coordination with Apache Zookeeper - A Java Approch
Apache Zookeeper is an Open source framework which can be used for distributed coordination. To understand what Apache zookeeper does we will have to go back to the Concurrent Programming class we did at university. Think how we tried to do coordination among different Threads or processors  We used global variables most of the times.

Actually there are two ways to achieve coordination among processors

1) Message Passing
2) Shared Memory approach.


Now lets bring that problem to the next level. How to coordinate threads /processors that are running in different machines/nodes in a distributed system.  The above two approaches are still the solution for this level as well. (Yes its a pattern in science :) )

There are lot of messaging frameworks and  distributed shared memory abstractions available which can be used to achieve distributed coordination.

I'd like to think Apache zookeeper as a Coordination framework which gives a shared memory abstraction to the distributed processors by giving a global file system like data structure and operations to manage it. And also it will provide additional set of features that will ease the developers life who will write coordination algorithms like being able to subscribe for node changes etc...

Anyway bottom line is its just a global data structure that all processors/nodes will see. No MAGIC!!! You will have to write distributed coordination algorithms by yourself  if you need to do coordination.

I was working on an effort on implementing Distributed Message Broker in which worked on creating some distributed coordination algorithms

We were using Apache zookeeper to handle some of the coordination tasks. Most of the coordination algorithms are specific to the problem domain we were working on like queue worker distribution balancing the load , handling node failures , node_id coordination etc..  But in this blog i’ll provide an example with a java code on how to implement a distributed resource lock using zookeeper. Which is one of the simplest and common use cases.  Which i think is a good starting point for Java programmers who wants to start working with Zookeeper.

This is a generic problem and there is a nice recipe provided by zookeeper http://zookeeper.apache.org/doc/trunk/recipes.html. But it may be hard to understand in the first read. Even i had to explain it to few people on how it works. So i’ll explain the algorithm first and then shoot the code.

Following are the steps of the algorithm http://zookeeper.apache.org/doc/trunk/recipes.html#sc_recipes_Locks)

    Call create( ) with a pathname of "_locknode_/guid-lock-" and the sequence and ephemeral flags set. The guid is needed in case the create() result is missed. See the note below.
    Call getChildren( ) on the lock node without setting the watch flag (this is important to avoid the herd effect).
    If the pathname created in step 1 has the lowest sequence number suffix, the client has the lock and the client exits the protocol.
The client calls exists( ) with the watch flag set on the path in the lock directory with the next lowest sequence number.
if exists( ) returns false, go to step 2. Otherwise, wait for a notification for the pathname from the previous step before going to step 2.

Explaining more about the steps in the step one We create a node with name like *guid*-lock-0001 where guild is a unique id generated by the client which can be used by the client to later identify the node it created. The number at the end is the sequence number appended by zookeeper since we set the flag sequential . And also one thing to note is we have also set the ephemeral  flag when creating the node. So that means this node will automatically get deleted once the client connection which created that node terminates.

In the steps 2 and 3 what we do is we get the all the lock nodes. and if our node is the node with the lowest sequence number(we can use guid to determine which node is created by us) that means we have the lock. So we can do our job and exit  (Note that when we exit , our connection to zookeeper terminates and it will delete the node created by our client)

If our node is not the one with lowest sequence number our client will reach the 4th step in the algorithm. Which means they do not have the lock and they must wait till they acquire the lock.

So in the 4th step what we do is we do a exist() call on the node with the next lowest sequence number (to current node, not the 2nd lowest number in the node set ) and register a watch for that. Idea is to avoid one node being busy handling watches. So after that each client will be watching only one node and one node is watched by only one client.

See the image bellow. Here N1.. N4 are nodes created by clients and here N1 holds the lock.




So once the N1 release the lock (Since the client connection terminates ) that N1 node will be deleted which  will be seen by N2 and since it rerun the algo from step2 it will be selected as the  lock holder.

Here someone will ask a fair question , what if a client that created N3 dies ? That case also handled by this algorithm too. Say N3 Client dies so in that case N4 Will be notified and it will go to step 2 in the algo which will get All the children and relaise its not the leader and then it will again set a watch to N2 which the the next lowest node to it .

The below picture shows what happen in this scenario.




after N4 run the algo from Step2 it will look like this.




So having this understanding let’s go into implementation details. So 1st thing we need to have is a API design for the Lock.

In this case i’ll implement this as a Class Named ResourceLock which will take Resource Name as a constructor argument and have 3 other methods

    acquire - acquire the lock the given resource. If lock is not available wait  till lock  acquired
    release - release the acquired lock
    destroy - clean up resource like connections etc.. 


Following is a Sample Java implementation of this Lock I wrote. In this case i wanted to make this lock a reusable one within a application across multiple threads instead of creating and exiting zookeeper connection each time i reused it. So when a client release lock it will delete that node from Zookeeper  instead of terminating the connection. I used Java Semaphores to enforce the behavior that is given to the outside world by the API as described above

Resource Lock Class :

package org.wickramarachchi.blog;

import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;
import org.wso2.zookeeper.sample.leader.coordination.CoordinationException;

import java.util.HashMap;
import java.util.List;
import java.util.UUID;
import java.util.concurrent.Semaphore;

public class ResourceLock {

  private final Object lock = new Object();

  private String myNode;
  private String myZNode = null;
  private int myId;

  private ZooKeeper zooKeeper = null;

  private String address;
  private int port;
  private String resource;

  public static String RESOURCE_LOCK_PARENT ="/resource_lock_parent";

  public static String NODE_SEPARATOR = "/";


  public static String RESOURCE_LOCK_NODE = "/resource_lock_node";

  /**
   * Creates a Distributed Lock for a resource
   *
   * @param address  zookeeper instance host name
   * @param port     zookeeper instance port
   * @param resource resource name
   */
  public ResourceLock(String address, int port, String resource) {

      this.address = address;
      this.port = port;
      this.resource = resource;

  }


  /**
   * Acquire the lock for the resource. This will get blocked till the it get
   * the lock for the resource
   *
   */
  public void acquire() throws InterruptedException, CoordinationException {
      try {

          if (zooKeeper == null) {
              synchronized (lock) {
                  if (zooKeeper == null) {
                      System.out.println("Starting Zookeeper agent for host : " + address + " port : " + port);
                      zooKeeper = new ZooKeeper(address, port, null);
                      System.out.println("ZooKeeper agent started successfully and connected to  " + address + ":"
                              + port);
                      try {
                          if (zooKeeper.exists(RESOURCE_LOCK_PARENT + "_" + resource,
                                  false) == null) {
                              zooKeeper.create(RESOURCE_LOCK_PARENT + "_" + resource,
                                      new byte[0],
                                      ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
                          }

                      } catch (Exception e) {
                          String msg = "Error while creating Queue worker coordination parent at " +
                                  RESOURCE_LOCK_PARENT + "_" + resource;
                          e.printStackTrace();
                          throw new CoordinationException(msg, e);
                      }
                  }
              }
          }
          createNode();
          proceed();


      } catch (Exception e) {
          throw new CoordinationException("Error Acquiring Lock ", e);
      }


  }


  /**
   * Release the Lock for the resource
   *
   * @throws CoordinationException
   */
  public void release() throws CoordinationException {

      try {
          deleteNode();
      } catch (Exception e) {
          throw new CoordinationException("Error while releasing lock", e);
      }

  }


  private void createNode() throws InterruptedException, KeeperException {


      final String nodeName = RESOURCE_LOCK_NODE +
              (UUID.randomUUID()).toString().replace("-", "_");
      this.myNode = nodeName.replace("/", "");
      String path = RESOURCE_LOCK_PARENT
              + "_" + resource + nodeName;
      zooKeeper.create(path, new byte[0],
              ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);

  }


  private void deleteNode() throws InterruptedException, KeeperException {
      if (zooKeeper != null) {
          String path = RESOURCE_LOCK_PARENT + "_" + resource +
                  NODE_SEPARATOR + myZNode;
          zooKeeper.delete(path, -1);
      }
  }


  private List<String> getChildren() throws InterruptedException, KeeperException {

      return zooKeeper.getChildren(RESOURCE_LOCK_PARENT + "_" + resource, false);
  }


  private boolean proceed() throws InterruptedException, KeeperException {

      while (true) {
          final Semaphore lock = new Semaphore(1);
          lock.acquire();
          List<String> childNodes = getChildren();
          HashMap<Integer, String> nodeIdMap = new HashMap<Integer, String>();

          String selectedNode = null;
          int currentMin = Integer.MAX_VALUE;
          for (String child : childNodes) {
              String id = child.substring(myNode.length());
              int seqNumber = Integer.parseInt(id);
              if (child.contains(myNode)) {
                  myId = seqNumber;
                  myZNode = child;
              }
              nodeIdMap.put(seqNumber, child);
              if (seqNumber < currentMin) {
                  selectedNode = child;
                  currentMin = seqNumber;
              }
          }

          assert selectedNode != null;
          if (selectedNode.contains(myNode)) {
              System.out.println("Lock acquired..");
              break;
          } else {
              int myLockHolder = --myId;
              Stat stat = zooKeeper.exists(RESOURCE_LOCK_PARENT +
                      "_" + resource + NODE_SEPARATOR +
                      nodeIdMap.get(myLockHolder),
                      new Watcher() {

                          @Override
                          public void process(WatchedEvent watchedEvent) {
                              if (Event.EventType.NodeDeleted == watchedEvent.getType()) {
                                  System.out.println("Locked Release Detected.. Trying to acquire lock again..");
                                  lock.release();

                              }
                          }
                      });

              if (stat == null) {
                  System.out.println("Locked Release Detected.. Trying to acquire lock again..");
                  continue;
              }

              lock.acquire();
          }


      }

      return true;
  }

  /**
   * Cleanup allocated Zookeeper Resources for this Lock
   */
  public void destroy() throws CoordinationException {

      try {
          zooKeeper.close();
      } catch (InterruptedException e) {
          throw new CoordinationException("Error while releasing the Queue Lock ", e);
      } finally {
          zooKeeper = null;
      }
  }

}




Test Class to Test this lock. In this case Clients from different threads try to access the same resource but Lock Make sure that one process have the resource at a given time.

package org.wickramarachchi.blog;



public class LockTest {


 public static void main(String[] args) throws Exception {
     new LockTest().run();
}


public void run() throws Exception{
    Thread t1 = new Thread(new Process(1));
    Thread t2 = new Thread(new Process(2));
    Thread t3 = new Thread(new Process(3));
    Thread t4 = new Thread(new Process(4));

    t1.start();
    t2.start();
    t3.start();
    t4.start();
}


class Process implements Runnable{


   int id;

   public Process(int id ) {
        this.id = id;
   }
    @Override
    public void run() {
        try {
            String resource= "resource";
            ResourceLock lock = new ResourceLock("127.0.0.1", 2181,resource);


            System.out.println(id + " Acquiring Lock" );
            lock.acquire();
            System.out.println(id + "get Lock" );

            Thread.sleep(500);

            System.out.println(id + " Releasing Lock" );
            lock.release();
            System.out.println(id + " Released Lock" );
            lock.destroy();

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}
}


by Charith Wickramarachchi at April 10, 2012 06:36 AM


Prabath Siriwardena—OWASP Sri Lankan chapter inaugural meeting...
During last couple of months I got the opportunity to getting involved in more community events.

This time it's the inaugural meeting of OWASP Sri Lankan chapter.

It's great to see such a forum being formed and being active. My session there was on 'Ethical Hacking'. Mostly I focused on to demonstrate how silly mistakes from programmers could lead to catastrophic security breaches.





Also there was another session on “Security in Your Own Way,” which was presented by Rosita De Rose, Process Lead – 99X Technology.  

All-in-all kudos the organizers - we need to keep this going...

You can read more details of the event from here...
by Prabath at April 10, 2012 04:24 AM


Ishani Ratnayake—Who is an Agile BA ?
Business Analysis (BA) is a role which collect, analyses requirements and get it across to the Engineers who build the ultimate product. It's crucial since it's the first step in any lifecycle be it product or software or any other. However, in  the traditional BA role, solution requirements are fixed and documented  up front on how the solution is expected to function at the end of the  project. But with the constant change in the industry and competing firms with each other in the industry demand ones' to be Agile. This is where the traditional role of a Business Analyst gets converted to an agile model. An Agile BA is a role who is involved through out  the entire life cycle of the project and involved in the facilitation  and management of evolving requirements.

I'm a Business Analyst working for WSO2, a open source application development software company focused on providing service-oriented architecture  (SOA) solutions for professional developers. On this blog post I would elaborate on my experience being a BA and challenges I faced during performing such. What I wanted to jot down this post is because personal experience be it good or bad, are the ones that drives you to the correct path. 

The value addition of engaging a Business Analyst arise by actually facilitating them to be a part of the interface for a client in to an Organization. It's vital that the BA folks talk to the client throughout the Client life cycle since then the person would understand the evolving requirements of the customer and apply the learnings for the new clients. With the constant change in the markets a BA is no longer a person who documents each and every move of the client neither a person who does all the documentation; but an individual who should make an impression on the client. 


Why is it exciting to be a BA ...


Being a BA is exciting since you get to learn so many different domains by dealing with clients. You not only get the wide exposure dealing with different clients but you tend to learn a lot since all those clients who deal contains are the best in their own domain. 


We are thrive to make relationships with the client and not only just help sort out their identified problem. Sometimes these relationships help tone your own carrier.


For me, I loved the fact that how I saw technology at WSO2 is being put to solve a middle-ware problem. At WSO2 we have a complete PaaS platform and it's pretty technical since its' end user is always a developer itself. Therefore, the fact that how to apply the knowledge I have gathered testing the products being a QA in my carrier I now see how actual clients use them for different use cases. I used to look deeper in to technical back then but now I look at them in a birds eye viewpoint and how each technologies map in to an architecture. Its quite fascinating. 


Other side of the Coin...

Well, it's by nature that every job has its own challenges. But it's important for any job out there to define the boundary. Most of the people fail to do their jobs due to a couple of reasons, either they dont know what they are expected to do, or there's no clear responsibility or authority provided or someone else is overriding the task that you do or lastly the individual doesn't have the required skills.

For any BA including myself have faced or still facing almost all.  

Why is it important for a BA to stick throughout the life cycle is; obviously  from one call one cannot get the Business Problem they are trying to  solve unless the client is really talkative :) Some don't just spill them  out or some may just be 'so' vague and it takes forever to understand  where the problem is. Working for a middle-ware company during the past 8 months or so, whenever I questioned about the business problem all they say told is they want to move in to a SOA. Is this the business problem? Hell No !! It's a step they just took to solve the problem. Of course if you just want to finish off the business that answer is more than adequate as we have a complete PaaS to cater to any middleware requirement that is out there. OK... The next problem is what is a business problem. For example, Lets say you're an e-ticketing company. Having the time taken between an end user clicks on a website and the time the end user gets the response is high. That is a problem !! It makes a problem since the end users will refrain from using the site if the site is slow. Well now one may say that for such a problem the how we find the root cause? Since the delay could arise anywhere. But making note of such a problem ensures that the problem is solved. If the ultimate problem is not solved if they choose the technology they thought, this would provide a bad impression to the technology provider. Whatever you do it's equally important that you do the RIGHT thing although you may loose the business. Being shortsighted will get the deal closed off but the company would not sustain. Hence the most important role of a BA is to identify the Business Problem and get the relevant architects to solve the problem they face.

Not knowing what to do is dangerous and this may be due to the lack of responsibilities you are put up with. How do we overcome this? Guess it's partly the task of the management of any organization to clear the boundary of a BA to perform its task and be very clear what they expect out of them. It'll be conflicting since a BA identifies the problem and this may be at the expense of the time. But it's crucial from the clients point of view. So at any given point of time there will be a cold battle between a sales guy whose looking to get to the target vs a BA who slows things down.  

Since you are one of the very first guys the client talks it is important that you know your basics to surprise the client. In a good way :) Sometimes it's impossible to join in every engagement with the client, but a subsequent meeting should always be held to brief a BA of the technologies they have to identify and promote any remedies for potential problems similarly any new developments in the company. Also, a BA should learn the basic architectures with the help of Solution Architects since almost the same thing just move and evolve around. Knowing such most used architects makes you explain the solution to the customer easily and with real world examples the client will buy it. Therefore there should be frequent meetings to brief the BAs of such situations the technology has made a significant difference onsite.

All in all, if the BA be only put as a documentation guy a company will loose investing on such a role. Therefore, it's crucial to correct it at an early stage by clearing out the boundaries and identifying the actual responsibility of such a figure.
  



by Ishani at April 10, 2012 02:20 AM

April 06, 2012


Kathiravelu Pradeeban—The GSoC community activities in Sri Lanka
We had one more session on GSoC today at the IT faculty of the University of Moratuwa. Only 30 participants were there, and it was a small scale one compared to the previous ones (150 heads was recorded as the highest in our GSoC sessions history - which was the session we had last week at CSE).



This year we have done several GSoC sessions, with the presentation crew consists of KasunG, Nirmal, Keheliya, Suho, Eranda, Shelan, Subash, Thilanka, and myself.



1) At the Engineering Faculty of the University of Peradeniya (invited and coordinated by the students).

2) At the Science Faculty of the University of Jaffna (invited directly by the staff of the Computer department, UoJ).

3) At the Apache BarCamp Colombo, at the University of Colombo, School of Computing.

4) At the department of Computer Science and Engineering, University of Moratuwa  (coordinated by the students, with the involvement and presence of the staff) - This event is also blogged in the Google Open Source Office's blog.

5) At the IT Faculty, UoM (coordinated by the students).



Except the session we had at the Apache BarCamp, all the other 4 GSoC sessions were complete GSoC sessions, and was organized by the individual volunteers, without any sponsors or involvement of other entities. We felt true passion of the speakers and the audience in this.





In past years, we had just one or two sessions. But this time, we made it to 5, and truly made it island-wide. It was a great experience presenting my introductory presentation to the Google Summer of Code, focusing the the open source way of communication, GSoC fundamentals, and the culture, for 80 mins in each of these sessions. Sharing knowledge and giving back to the community is a pleasant feeling; thanks everyone involved in each of these events. All of them are awesome!

by Kathiravelu Pradeeban at April 06, 2012 08:50 AM

April 05, 2012


Afkham Azeez—WSO2 Load Balancer - how it works



... an under the hood look at the WSO2 Elastic Load Balancer.















In The role of a Load Balancer in a Platform-as-a-Service, I explained some key concepts behind load balancing. In this article, we will take a look at the inner workings of the WSO2 Elastic Load Balancer to understand how some of these concepts have been incorporated. The WSO2 Elastic Load Balancer is the load balancer used in StratosLive, the first fully complete open Java Platform-as-a-Service (PaaS)






High Level Architecture

















As shown in the diagram above, the WSO2 Elastic Load Balancer is built using Apache Tribes, which is the group management framework used by Apache Tomcat as well as Apache Axis2, Apache Axis2 clustering module, Apache Synapse - one of the best performant mediation frameworks, and the autoscaling component from the award-winning WSO2 Carbon framework.  The autoscaling component interacts with the Amazon EC2 API to carry out infrastructure related functionalities such as starting new instances, terminating running instances, mapping elastic IPs & so on.




Let's start looking at the above diagram from the topmost layer. The Carbon autoscaling component is responsible for keeping track of the traffic to each Cloud Service cluster, and making decisions related to scaling the system up when the load increases & scaling the system down when the load reduces. It is also responsible for keeping a sanity check on the system. This sanity check ensures that the minimum system configuration, such as the minimum number of running instances of each Service & load balancer cluster is maintained at all times. This autoscaling component calls out to the EC2 API for infrastructure related activities. The autoscaling component has been implemented as two Apache Synapse mediators; autoscale-in & autoscale-out mediators, and a Synapse task.




A Synapse endpoint called the ServiceDynamicLoadBalanceEndpoint is responsible for routing the messages to the appropriate node. First, the Cloud Service cluster to which the message has to be sent is identified using the Host HTTP header. Then a member from that Service cluster to which the message has to be routed to is selected according to the load balancing algorithm specified, and the message is sent to that member. The response from that member is sent back to the client which originated the request. While the LB is trying to forward a request to a member, that member may fail, and in such a case, if possible, the ServiceLoadBalanceEndpoint will try to failover to another available member. Such a member which failed will be suspended for a specified period. This endpoint is also responsible for handling sticky sessions. In fact, when a request comes in, this endpoint first checks whether there is sticky session created for that client, and if such a session is found, the request is forwarded to the relevant member. Sticky sessions are identified using the value of the JSESSIONID cookie.




Binary Relay or Message Relay is the message pass through mechanism in the WSO2 Elastic Load Balancer. This relay enables the LB to pass the messages without building or processing the message. This ensures that the overhead introduced by the LB is minimal.




Membership discovery & management is handled by the Axis2 Clustering module, which handles Service membership. There are multiple membership handlers, one for each clustering domain, responsible for handling membership. The ServiceDynamicLoadBalanceEndpoint will obtain the relevant members of the cluster from this Axis2 membership handlers, and then apply the load balance algorithm to determine the next member to which the request has to be sent. The Axis2 clustering implementation uses the underlying Apache Tribes Group Management Framework for group membership management.




Dynamic Clusters & Group Management Agents





















In the WSO2 Elastic Load Balancer, we have support for dynamic & hybrid membership discovery. As shown in the above diagram, the Elastic Load Balancer can be set up in a primary-secondary configuration. These LBs will be in a cluster, and state replication can take place between these nodes. The LBs are also configured as Group Management Agents, hence they can manage domains A, B & X. This means, any membership changes in the domains, A, B, X will be visible to LBpri & LBsec. A public IP address will be assigned to the LBpri in a typical deployment. The LBsec will keep observing LBpri, and if LBpri fails, LBsec will map the public IP to itself, spawn another secondary LB, and then become the primary LB. The client sees the public IP address mapped to LBpri, and needs to send the HTTP 1.1 Host header. This Host header will be used by LBpri to determine the destination Service cluster. For example, if the Host is set as Hb, the LBpri will select one of the nodes in domain B, and send the request to it.




For this setup to work, clustering has to be enabled in all the backend worker nodes, as well as the load balancers, because the underlying Axis2 clustering mechanism & Tribes group management framework are used for dynamic membership management.




Real Port to Proxy Port Mapping














The mapping between the proxy port & the real port on the member (worker node) is provided as shown in the diagram above. In the above diagram, we have shown an example of the LB exposing 4 ports; 2 for HTTP & 2 for HTTPS requests. The HTTP ports on the LB are 80 & 8280. The HTTPS ports on the LB are 443 & 8243.




Member A1 exposes 4 ports; HTTP 9762 is proxied via port 80 in the LB, HTTP 9763 is proxied via port 8280 in the LB; HTTPS 9443 is proxied via port 443 in the LB & HTTPS 9444 is proxied via port 8243 in the LB. 




Member A2 also has a similar real port to proxy port mapping as depicted in the diagram above.




These ports & the mapped proxy ports will be advertised by the members when they join the cluster, and the LB can retrieve these values as properties of the member. When the LB receives a request, it will get the incoming port, and before dispatching the request to a member, it will try to get the mapped real port from the member, and route the request to the appropriately mapped port (as advertised by the member).




The diagram shown below shows how requests from clients will be proxied from the LB to member A1.














How Elasticity/Autoscaling is Handled

Autoscaling is handled by the WSO2 Carbon autoscaling component. This component keeps track of the number of messages in flight to each Service cluster, and decides whether to scale the system up or down. The autoscaler component consists of an AutoscaleIn mediator, AutoscaleOut mediator & LoadAnalyzerTask, as shown in the diagram below. When a message is received, the AutoscaleIn mediator creates a unique token & puts it into a list. When a response to the message is sent, the AutoscaleOut mediator removes that token from that list. So, this list tracks the number of messages in flight for each backend Service. The LoadAnalyzerTask periodically check the list lengths & based on the configuration parameters, decides whether to scale up or scale down a backend Service.

















This component will start new Service member instances, and once those members successfully boot up, they will join the relevant Service cluster. Now the load balancer will start forwarding the request to the new members as well.




Membership Management using Apache Tribes














As shown in the above diagram, the WSO2 Elastic Load Balancer is a special Tribes member, which can sense membership changes in the groups it is managing. Each group has its dedicated membership channel, and the LB can connect to all these channels. The other Service members can only see membership changes in their respective channels. In the above diagram, members in Group A (A1, A2, A3,..., An) can sense membership changes in Membership Channel - A, but cannot see such changes in Membership Channel -B. However, the LB can see membership changes in Channels A, B & LB.




Deployment in a Data Center

If your network allows multicasting, you could use the multicast based membership scheme. In this case, membership will be handled using multicasting, hence a multicast socket needs to be configured.




Deployment on the Cloud

On a Cloud deployment, typically you would use well-known address based membership scheme since multicasting will be disabled on such setups. In such a setup, we will make the one or more LBs to be the well-known members. All nodes in the Service clusters, SA, SB & SX will see the LB as a well-known member. Membership will revolve around the well-known load balancer in this setup.




WSO2 Elastic Load Balancer on StratosLive














The above diagram shows how we have deployed the WSO2 Elastic Load Balancers in StratosLive. The diagram shows how a single LB instance can front several Cloud Service clusters. For example, the Manager, Business Rules Server (BRS) & Business Process Server (BPS) Cloud Services are fronted by a single load balancer, and the Governance Registry (G-Reg), Mashup Server (MS) & Gadget Server (GS) Cloud Services are fronted by another load balancer. Some other services such as the Identity Server, Application Server, Business Activity Monotor (BAM) & Data Services Server (DSS) have dedicated load balancers, since these are popular Services with heavy incoming traffic. As you may note, some nodes are marked as permanent nodes. This means, these nodes are not supposed to be shutdown if the autoscaler makes a scale-down decision. Special flags mark these nodes as permanent nodes, and the autoscaler will check this flag before deciding to terminate an instance. It is the elastic nodes that can be terminated if a scale-down decision is made.





by Afkham Azeez at April 05, 2012 10:38 AM


Kathiravelu Pradeeban—How Auto Scaling works with WSO2 Load Balancer
The autoscaling component comprises of the synapse mediators  AutoscaleInMediator and AutoscaleOutMediator and a Synapse Task  ServiceRequestsInFlightEC2Autoscaler that functions as the load analyzer  task. A system can scale up based on several factors, and hence  autoscaling algorithms can easily be written considering the nature of  the system. For example, Amazon's Auto Scaler API provides options to  scale the system with the system properties such as Load (the timed  average of the system load), CPUUtilization (utilization of the cpu at  the given instance), or Latency (delay or latency in serving the service  requests). 
Autoscaler Components

AutoscaleIn mediator - Creates a unique token and puts that into a list for each message that is received.
AutoscaleOut mediator - Removes the relevant stored token from the list, for each of the response message that is sent.
Load Analyzer Task -  ServiceRequestsInFlightEC2Autoscaler is the load analyzer task used for  the service level autoscaling as the default. It periodically checks the  length of the list of messages based on the configuration parameters.  Here the messages that are in flight for each of the back end service is  tracked by the AutoscaleIn and AutoscaleOut mediators, as we are using  the messages in flight algorithm for autoscaling.




ServiceRequestsInFlightEC2Autoscaler   implements the execute() of the Synapse Task interface. Here it calls   sanityCheck() that does the sanity check and autoscale() that handles   the autoscaling.


Sanity Check
sanityCheck()   checks the sanity of the load balancers and the services that are load   balanced, whether the running application nodes and the load balancer   instances meet the minimum number specified in the configurations, and   the load balancers are assigned elastic IPs.


nonPrimaryLBSanityCheck()    runs once on the primary load balancers and runs time to time on the   secondary/non-primary load balancers as the task is executed   periodically. nonPrimaryLBSanityCheck() assigns the elastic IP to the   instance, if that is not assigned already. Secondary load balancers   checks that a primary load balancer is running periodically. This avoids   the load balancer being a single point of failure in a load balanced   services architecture.


computeRunningAndPendingInstances()   computes the number of instances that are running and pending.   ServiceRequestsInFlightEC2Autoscaler task computes the running and   pending instances for the entire system using a single EC2 API call.   This reduces the number of EC2 API calls, as AWS throttles the number of   requests you can make in a given time. This method will be used to  find  whether the running instances meet the minimum number of instances   specified for the application nodes and the load balancer instances   through the configuration as given in loadbalancer.xml. Instances are   launched, if the specified minimum number of instances is not found.
Autoscale
autoscale()   handles the autoscaling of the entire system by analyzing the load of   each of the domain. This contains the algorithm - RequestsInFlight  based  autoscaling. If the current average of requests is higher than  that can  be handled by the current nodes, the system will scale up.  If  the  current average is less than that can be handled by the (current  nodes -  1), the system will scale down.


Autoscaling  component spawns new instances, and once the relevant services  successfully start running in the spawned instances, they will join the  respective service cluster. Load Balancer starts forwarding the service  calls or the requests to the newly spawned instances, once they joined  the service clusters. Similarly, when the load goes down, the  autoscaling component terminates the under-utilized service instances,  after serving the requests that are already routed to those instances.

by Kathiravelu Pradeeban at April 05, 2012 10:07 AM



Kathiravelu Pradeeban—StratosLive - A case study for WSO2 Load Balancer
In a cloud environment such as WSO2 StratosLive, auto-scaling becomes a crucial functionality. The system is expected to scale up and down with the dynamically changing load. Auto-scaling capabilities are sometimes provided by the Infrastructure as a Service provider themselves, such as the Autoscaling from Amazon. However, autoscaling is not necessarily a requirement that to be fulfilled by an IaaS. Say, you are providing Platform as a Service (PaaS) that is hosted over the pure native hardware, instead of an IaaS. In that case, your PaaS should be able to provide the required autoscaling and load balancing capabilities to the applications that are hosted on top of your platform. WSO2 Load Balancer is such a software load balancer, that handles the load balancing, fail over, and autoscaling functionalities.



WSO2 Load Balancer  is used in production as a dynamic load balancer  and autoscaler, as a  complete software load balancer product. It is a  stripped down version  of WSO2 Enterprise Service Bus, containing only  the components that are  required for load balancing. WSO2 StratosLive  can be considered a user  scenario with WSO2 Load Balancer in  production. 





Multiple  service groups are proxied by WSO2 Load Balancers. Some of the services  have more than one instances to start with, to withstand the higher  load. The system automatically scales according to the load that goes  high  and low. WSO2 Load Balancer is configured such that the permanent  or the initial nodes are not terminated when the load goes high. The  nodes that are spawned by the load balancer to handle the higher load  will be terminated, when the load goes low. Hence, it becomes possible  to have different services to run on a single instance, for the  instances that are 'permanent', while the spawned instances will have a  single carbon server instance.

by Kathiravelu Pradeeban at April 05, 2012 09:42 AM

April 04, 2012


Chris Haddad—Pinterest API and the Money Board
Pinterest is being touted as a web ecosystem platform that may potentially rival FaceBook.  To be a web ecosystem platform, an online website property publishes an API enabling an ecosystem of 3^rd developers.   Once Pinterest publishes an API, 3^rd party developers could extend core Pinterest functionality, enrich the user experience, and accelerate user adoption.    According to recent reports, Silicon Valley entrepreneurs and investors are ready to ride user interest in Pinterest:
“When the API is available, Adam Ludwin at RRE, says he’s interested in entrepreneurs who can build on top of the platform, who can “close the loop” and figure out a way to monetize the interest around products on Pinterest. He’s interested in startups that will provide tools to facilitate transactions on and through Pinterest.”
But exposing an API can lead other companies capturing revenue otherwise directed to Pinterest.  As reported recently by Jay Yarow, Pinterest
“might not release it [API] for a while, says an industry source familiar with Pinterest’s plans. This source says that Pinterest fears having a ‘Twitter problem.’”

What is the Twitter problem?
Twitter, the widely successful ‘open data’ messaging company, recently encountered head-on conflict with its ecosystem partners.   Twitter’s open API and liberal terms of service yielded numerous ‘partnering’ organizations, who extended Twitter and made the basic service more usable through 3rd party applications.  The problem for Twitter?  Twitter didn’t have mechanisms in place to monetize their user base, had their customer ownership diluted, and the company increasingly saw external companies making money from their extensions.  The solution, Twitter embarked on a costly and time consuming defensive strategy to buy Twitter clients, locked down access to their API, and disenfranchised the 3^rd party developer community.
Om Malik of GigaOm rightly points out that the Twitter’s easy access policies were based on not defining their business model and then making appropriate decisions.  Had Twitter been aware of its options and consequences from the early days, they may have defined a clearer ecosystem strategy. For example, Om states
“If Twitter at some point thought of itself as a media network, then its business model option would have included controlling the front end(s) to the service. Instead, it didn’t do any of those things. One logical explanation is that the service itself was evolving as it went along.”
While Pinterest is playing smart by moving cautiously, how should Pinterest effectively monetize user interest and create an engaging ecosystem?
An Ecosystem Platform Recommendation for Pinterest
With data APIs, users are always one step removed from the platform company. Intermediary applications (between the data provider and end user) have the opportunity to reshape the customer relationship.  Rather than rely solely on data APIs, we have seen forward-thinking organizations create ecosystem platforms revolving around user experience APIs and domain specific hosting environments. With user experience APIs, the platform company can create an ‘Apple experience’; maintaining customer ownership by controlling the ‘look and feel’ and by authorizing third party extensions.  Add in a 3^rd party application hosting environment‘ (similar to SalesForce.com), and the ecosystem platform can manage Quality of Service (QoS), share monetization revenue, and enforce compliance regulations (e.g. PCI Security Standards, HIPAA, European Union Directive on Data Protection of 1995).
Instead of simply publishing data APIs, Pinterest could manage and host 3^rd party applications within a multi-tenant cloud ecosystem platform.  WSO2 clients are using WSO2 API Management and WSO2 AppFactory, a cloud ecosystem platform, to deploy context-aware APIs, rapidly provision 3^rd party application projects, automate governance approval tasks, ensure regulatory compliance, monetize user interactions, and host applications that seamlessly extend the user experience.
The ecosystem platform environment also provides an opportunity for Pinterest’s partners to deeply embed their business capabilities within Pinterest’s application (similar to Force.com, eBay sellers, or Amazon Store environments).  By hosting all business partners as tenant applications within a multi-tenant environment, the ecosystem environment more readily aggregate and share business information.  Figure 3 illustrates how a complete middleware platform, API management, and Platform as a Service can be composed as an ecosystem platform.




Figure 1: Ecosystem Platform Environment

Bottom Line
Pinterest is playing smart by moving cautiously and fully evaluating business models and core features before opening up their platform.  To effectively monetize user interest and create an engaging ecosystem, we recommend Pinterest create ecosystem platforms revolving around user experience APIs and a domain specific hosting environment.  WSO2 AppFactory, a cloud ecosystem platform, will enable Pinterest to deploy context-aware APIs, rapidly provision 3^rd party application projects, automate governance approval tasks, ensure regulatory compliance, monetize user interactions, and host 3^rd party applications that seamlessly extend the user experience.
 
Related Posts and References
WSO2 API Management Platform
APIs and Beyond
Creating and Ecosystem Platform with Vertical PaaS
 
 

by haddadcblg at April 04, 2012 11:33 AM


April 03, 2012


Hasini Gunasinghe—SCIM Interop Event at IETF 83rd Meeting
First interop event organized by SCIM working group (or originally named as Cloud Directory WG), was held in the room: Corot of hotel Concorde La Fayette, Paris on 28th of March from 6-12 CET. 

Purpose: The purpose of this event was to bring together current working implementations of SCIM and test the level of interoperability between each other which in turn could be used as a valuable input to prove the interoperability of the SCIM spec itself for the Birds of Feathers Session  that was held on 29th of March, in order to form a SCIM working group in IETF.

SCIM (Simple Cloud Identity Management) is an emerging standard that is focused on identity provisioning. You may refer to my previous post to get an overview of SCIM. 

Participants:  
Erik & Samuel from Technology Nexus, 
Kelly from Sailpoint, 
Chuck from Salesforce, 
Trey from UnboundID, 
Travis from Ping Identity, 
Morteza from Cisco, 
Emmanuel from BCPSOFT,
Hasini from WSO2,
participated in person while Michael  from Gluu and James from Curion, participated remotely.

Following are some pics I took during the interop event:



Rest of the post is mainly about the interop experiences of WSO2 Charon when it was tested against SCIM service providers and SCIM Clients provided by other implementations.

WSO2 Charon:
Charon is the open source SCIM implementation offered by WSO2 under Apache 2.0 license. You may refer to my previous blog posts to get an idea about WSO2 Charon. Milestone 1 of WSO2 Charon was released in the time for first interop event. You may refer to a nice blog post written by Prabath on M1 release of Charon. We hosted a public SCIM endpoint for interop testing at people.wso2.com. 

At the start of the interop event, Every one shared their server credentials with participants and started testing by picking one endpoint at a time and sending requests from their clients. 

WSO2 Charon SCIM Client was tested against the SCIM endpoints provided by following SCIM service provider implementations:
Technology Nexus
UnboundID
Curion
Salesforce
Ping Identity  
This list doesn't contain all in the participants list above, since Sailpoint offers only a SCIM Client implementation and other service provider endpoints were busy with fixing some of the issues encountered during the interop event.

Varying level of successes achieved when Charon client tested with each of the above endpoints among which highest percentage of success was achieved with Unbound Identity(8 out of 10 scenarios passed) and Technology Nexus(6 out of 10 scenarios passed) endpoints. 

WSO2 Charon-Samples module includes sample SCIM clients which cover all the SCIM operations supported by Charon as of its M1 release. These sample SCIM client programs made it easy to cover all the other working SCIM server endpoints during the 6 hours time period of the interop event.

Interop issues found:  Following are some of the issues found when testing WSO2 Charon client with other server endpoints which caused some operations to fail. We negotiated and discussed about how to align the implementations with the spec in order to overcome those issues.

1.Server expects ETag when update and delete requests are sent from client side. While it is a good feature to support resource versioning in server side, it should not be mandatory for client to set it according to the spec. Therefore agreed that server side should also allow handle the requests which doesn't contain ETag header.

2. Server returns an error when read-only attributes are contained in the payload of an update request. According to the spec: "Consumers must retrieve the entire Resource and PUT the desired modifications as the operation overwrites all previously stored data." The example payloads in the spec also contains read-only "id" attribute in the update request. Therefore, we agreed that spec needs more clarifications whether it should fail or ignore the read-only attributes in the update request payload and update only other attributes.

3. Server had an internal requirement to include a group attribute when creating a user and if a group is not provided, create user operation fails. But according to the spec, group attribute in User resource is read-only. Therefore, we agreed that servers should not mandate to expect group attribute in create User request payload, even if they have internal server requirements to do so.

4. Server replies with dateTime attributes formatted in .NET DateTime strings. WSO2 Charon client expects date time attributes be formatted in XML Schema Datatypes Specification (2008-01-23T04:56:22Z) which is mentioned in SCIM spec. Agreed to follow the same format for dateTime as specified in SCIM spec in order to avoid interop issues even on these minor areas.

5. Server doesn't add an id attribute rather considers external ID as the id of the resource. Still it is not made mandatory to make it unique which may cause issues in retrieving a particular resource. 

Other SCIM Clients were tested against WSO2 Charon SCIM service provider endpoints:
Curion
Technology Nexus 
This list too doesn't contain all in the participants list above, because it took participants quite some time to test against one endpoint and the duration of the interop was 6 hours.

Note: WSO2 Charon endpoint is available for public access and you may carry out interop testing with it anytime and please let us know if you find any issues.

Interop issues found: Following is the only issue reported by the ones who tested against WSO2 Charon endpoint:

1. List user operation returning "resource not found" error.
This operation returns proper response with the list of resources like below, when List user operation is performed with WSO2 Charon client. :
{
"schemas":["urn:scim:schemas:core:1.0"],
"totalResults":2,
"Resources":
 [
  {
   "id":"0f6fd995-38fb-4240-a5ce-961a7032427f",
   "externalId":"umesha",
   "meta"{
          "lastModified":"2012-03-28T05:56:32",
          "created":"2012-03-28T05:56:32",
          "location":"http://localhost:8080/charonDemoApp/scim/Users/0f6fd995-38fb-4240-a5ce-961a7032427f"
          }
  },
  {
   "id":"e942ac6d-476c-4c7a-add3-f4ecb068a2f6",
   "externalId":"hasini@gmail.com",
   "meta":{
          "lastModified":"2012-03-28T05:53:27",
          "created":"2012-03-28T05:53:27",
          "location":"http://localhost:8080/charonDemoApp/scim/Users/e942ac6d-476c-4c7a-add3-f4ecb068a2f6"
          }
  }
 ]
}
But due to an interop issue when the same was performed with other clients, it shows an unexpected behavior which needs to be reproduced and identified with that particular client.

Overall Comments:
I consider the whole interop event was an effective session where we were able to discuss and agree on interpretation of certain points of  the spec wrt implementation aspects and identify areas of the spec  which needs more clarification in order to overcome some of the interop issues mentioned above. 

It was also a good community meetup where the people who are communicating remotely over the mailing list could get together and meet in person and make their implementations communicate with each other. 
I would like to Thank all participants for collaborating effectively during the session to make the interop event a success.

Now the SCIM spec has a new beginning at IETF after the successful BOF session which was held 29th of March 2012, and the standard will have a long journey to go till it is published as a RFC in IETF.

WSO2 Charon road map in brief:
WSO2 Charon will be feature completed with its 1.0 release and will be integrated into WSO2 Identity Server 4.0.0 so that WSO2 product stack and WSO2 Stratos will be equipped with the standardized identity provisioning feacture based on SCIM.

by Hasini at April 03, 2012 06:40 PM


Chris Haddad—An Enhanced User Experience with the WSO2 Mobile Services Gateway
To further improve the automobile shopping user experience, one of the largest clubs in the American Automobile Association Mid-Atlantic (AAA Mid-Atlantic) recently introduced the new AAA Auto Buying Tools mobile app. As the app design emerged, AAA Mid-Atlantic found a few obstacles in their way -


There was no straightforward means to facilitate communication between the mobile application and AAA’s services and data storage layers.
There was a need for better means of accessing and reusing existing services
There was a need to ensure flexibility in business rules without the need to continually redeploy the application.

Join Jordan Corn, Director of Solutions Research and Ed Klichinsky, Solutions Architect at AAA along with Asanka Abeysinghe, Director Solutions Architecture and Miyuru Wanninayaka, Senior Software Engineer at WSO2 as they discuss the implementation of a communication path between a mobile app (specifically an iPhone app) and AAA Mid-Atlantic. The session will commence with an introduction of the challenges faced by AAA in the advancement of the mobile app. This will be followed by a detailed discussion of how the WSO2 Mobile Services Gateway was successful in overcoming these challenges.
The recorded webinar presentation and associated slide deck can be found on the WSO2 Webinar archive page.
 
by haddadcblg at April 03, 2012 02:22 PM


Dakshitha Ratnayake—Logging with WSO2 ESB
Logging to the Server Log File
Add log mediators to your sequences in order to create log messages.

If you go through the sample index for WSO2 ESB and follow the steps on how to set up the samples  and then go to sample number 2 you will see how the log mediator is used.

e.g. -
<log level="custom">
        <property name="symbol" expression="get-property('symbol')"/>
        <property name="epr" expression="get-property('To')"/>
    </log>

For more information on the log mediator in ESB please check here.

Logging to Other Log Files
Log mediators log into the server logs, and if you dont want the application logs to write into server logs but to write to its own specific logs you can edit log4j.properties file and easily get your proxy services to log to separate log files.


Editing log4j.properties 
Simply add the following entries into your log4j.properties file of the ESB (this example assumes a proxy service named TestProxy):

log4j.category.SERVICE_LOGGER.TestProxy=INFO, PROXY_APPENDER
log4j.additivity.PROXY_APPENDER=false
log4j.appender.PROXY_APPENDER=org.apache.log4j.DailyRollingFileAppender
log4j.appender.PROXY_APPENDER.File=${carbon.home}/repository/logs/${instance.log}/wso2-esb-test-proxy${instance.log}.log
log4j.appender.PROXY_APPENDER.Append=true
log4j.appender.PROXY_APPENDER.layout=org.apache.log4j.PatternLayout
log4j.appender.PROXY_APPENDER.layout.ConversionPattern=%d{HH:mm:ss,SSS} [%X{ip}-%X{host}] [%t] %5p %c{1} %m%

With a configuration similar to the above you will see that TestProxy is now logging to a separate log file named wso2-esb-test-proxy.log as follows:

16:43:25,224 [-] [Framework Event Dispatcher] INFO TestProxy Building Axis service for Proxy service : TestProxy
16:43:25,225 [-] [Framework Event Dispatcher] INFO TestProxy Adding service TestProxy to the Axis2 configuration
16:43:25,669 [-] [Framework Event Dispatcher] INFO TestProxy Successfully created the Axis2 service for Proxy service : TestProxy
16:43:35,172 [-] [HttpServerWorker-1] INFO TestProxy To: http://localhost:8280/services/TestProxy, WSAction: urn:getQuote, SOAPAction: urn:getQuote, ReplyTo: http://www.w3.org/2005/08/addressing/anonymous, MessageID: urn:uuid:a3d87df8-6d24-4fda-b6ed-458906ff5e64, Direction: request, Envelope: http://localhost:8280/services/TestProxy</wsa:To><wsa:MessageID>urn:uuid:a3d87df8-6d24-4fda-b6ed-458906ff5e64</wsa:MessageID><wsa:Action>urn:getQuote</wsa:Action></soapenv:Header><soapenv:Body><m0:getQuote xmlns:m0="http://services.samples">IBM

Using Synapse Observers
Another alternative is to use synapse observers, which will be notified whenever a proxy service is called. We can write a custom observer that can grab proxy service level logs. For more information on observer classes go to http://synapse.apache.org/Synapse_Extending.html.

package org.wso2.carbon.customobserver.CustomSynapseObserverForLogging;

import java.io.IOException;
import org.apache.synapse.config.AbstractSynapseObserver;
import org.apache.synapse.core.axis2.ProxyService;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.log4j.DailyRollingFileAppender;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.apache.log4j.PatternLayout;


public class CustomSynapseObserverForLogging extends AbstractSynapseObserver {
      private static final Log log = LogFactory.getLog(CustomSynapseObserverForLogging.class);

       public void proxyServiceAdded(ProxyService proxy) {
               try {
                       setLogger(proxy);

               } catch (IOException e) {
                       log.error("CustomProxyObserver could not set service level logger for the proxy : " +
                                 proxy.getName(), e);
               }
       }

       public void proxyServiceRemoved(ProxyService proxy) {
               try {
                       setLogger(proxy);
               } catch (IOException e) {
                       log.error("CustomProxyObserver could not set service level logger for the proxy : " +
                                 proxy.getName(), e);
               }
       }


       private void setLogger(ProxyService proxy) throws IOException {

               String filename = "logs/" + proxy.getName() + ".log";
               String datePattern = "yyyy-MM-dd";
               String SYSTEM_LOG_PATTERN = "[%d] %5p - %x %m {%c}%n";

               PatternLayout layout = new PatternLayout(SYSTEM_LOG_PATTERN);

               DailyRollingFileAppender appender = null;

               appender = new DailyRollingFileAppender(layout, filename, datePattern);

               Logger proxyLogger = Logger.getLogger("SERVICE_LOGGER." + proxy.getName());

               proxyLogger.setLevel(Level.ALL);

               proxyLogger.setAdditivity(false);
               proxyLogger.addAppender(appender);

       }
}

1. In the log4j.properties file (located at ESB_HOME/lib) change the log level to DEBUG by setting log4j.category.org.apache.synapse=DEBUG (or any other level as preferred).

2. Edit synapse.properties (located at ESB_HOME/lib/core/WEB-INF/classes) by adding the following line.
synapse.observers=org.wso2.carbon.customobserver.CustomSynapseObserverForLogging

3. Complie the observer class and create a jar file that contains it and drop it to the ESB_HOME/repository/components/lib folder.
4. Run your application which calls a proxy service (or simply run sample number 150)
5. The log files will be generated at ESB_HOME/logs/*

All your mediator level logs will also be logged into these files.

by Dakshitha at April 03, 2012 08:51 AM

April 02, 2012


Ruchith Fernando—Gmail and Spam
When I tried to mark an email as spam Gmail prompted the following :




What I'm wondering now is : "Wouldn't Gmail's automated attempt to unsubscribe my email address from the spammer's list confirm the existence of a valid email address to the spammer?"
by Ruchith Fernando at April 02, 2012 02:59 PM


Chris Haddad—SCIM Simple Cloud Identity Management Interop
WSO2 Identity Server supports many leading identity management specifications, and work is underway to support interoperable Simple Cloud Identity Management.  The Simple Cloud Identity Management (SCIM) specification is designed to make managing user identity in cloud based applications and services easier.

It’s intent is to reduce the cost and complexity of user management operations and provide patterns for exchanging this schema using standard protocols. In essence, make it fast, cheap, and easy to move users in to, out of, and around the cloud. SCIM delivers more agile integration with Google Apps, Salesforce and other SaaS providers.
At a recent IETF meeting, WSO2, Sailpoint, Ping Identity, UnboundId, Cisco, Technology Nexus, SalesForce, Gluu, Curion, and BCPSoft performed interoperability testing.  The twenty-one SCIM interop use cases covered managing users and groups.  During the interop event, WSO2 Charon, WSO2′s SCIM implementation delivered under the Apache 2.0 license, performed well.  Interoperability testing uncovered a few areas where the specification requires clarification.
During the summer timeframe,  WSO2 will incorporate SCIM into the WSO2 Identity Server, a highly interoperable identity management platform.   WSO2 Identity Server delivers:

Single Sign-On (SSO) via OpenID, SAML2 and Kerberos KDC
Provisioning via SCIM instead of legacy SPML
Auditing via XDAS
Delegation via OAuth 1.0a, OAuth 2.0 and WS-Trust
Federation via OpenID, SAML2 and WS-Trust STS
Integration with Microsoft SharePoint with Passive STS support
Implement REST security with OAuth 2.0 and XACML
Attribute or Claim based access control via XACML, WS-Trust, OpenID and claim management
Fine-grained policy based access control via XACML

Let me know if you would like a demonstration of the only complete open identity management server based on:

Open Architecture
Open Standards
Open License
Open Source

 
by haddadcblg at April 02, 2012 02:00 PM


Yumani Ranaweera—Hierarchical service deployment support in WSO2 AS
Hierarchical service deployment support have been enabled in WSO2 App Server-3.2.0. The advantages of this feature are; 
Mainly you don't need to stick to a flat structure. This means you can manage different versions of the same service, where if only the business logic of the service is changed you can keep versions. Also it lets you more efficiency in managing your content.

Following gives lot of useful details;
 - Hierarchical service deployment support implementation
 - Hierarchical service deployment support practical example




by Yumani Ranaweera at April 02, 2012 11:36 AM


Yumani Ranaweera—Cheat-sheet : Starting WSO2-AS-4.2.1 with Activemq-5.2.2

Start Activemq

1. Download activemq from here
2. Extract the binary

3. Start activemq by going to ACTIVE_DIRECTORY_HOME/bin and issuing the command ./activemq console



Configure WSO2 AppServer
4. Download WSO2 AppServer from here


5. Extract the binary


6. Edit axis2.xml to enable JMS transport. 

Open CARBON_HOME/repository/conf/axis2.xml (CABRON_HOME is the location you extracted the AppServer distribution)
Enable JMS transport by uncommenting follwing sections

 
   -
   -
   -
 

AND
 


7. Copy following jars from ACTIVEMQ_HOME/lib to CARBON_HOME/repository/components/lib

activemq-core-5.4.2.jar
geronimo-j2ee-management_1.1_spec-1.0.1.jar
geronimo-jms_1.1_spec-1.1.1.jar

8. Also copy the same to CARBON_HOME/lib/api

9. Start the application server by issuing ./wso2server.sh at CARBON_HOME/bin.





by Yumani Ranaweera at April 02, 2012 11:36 AM

April 01, 2012


Madhuka Udantha—What is API management? (part 1)
Life consist of choices. Just now, I can watch move or call friend or write blog post. Recently I did not write one but I used to write three or five post per month. I missed it because I was developing something called 'API store' and 'API Provider' sites with wso2. It is interest topic by the way. I enjoyed it because it have nice features where we all needs. 
  If you never try out APIs or API management tool before. You will have a one question on your mind now. it will be 
  "What is API?"
  Application Programming Interface, is interface by software components to communicate with each other. An API specification can take many forms such as International Standard or vendor documentation API, or libraries of a programming or services. An API can be language-dependent or language-independent.
  service-oriented API are language-independent that is not bound to a specific process or system and we can remote procedure calls or web services. 
  For example, 
  I want to watch cricket match?
     So I have to find match details (when will it be? where will it be?)
    I need to buy ticket for the particular match 
    I need some transportation or navigation for that stadium 
    I will need some hotel for my accommodation 
 
  So what are the services that I am needing for this task to be achieved
     I have to find match details (when will it be? where will it be?)
 
  Cricket match services
     I need to buy ticket for the particular match 
 
  Ticket services
     I need some transportation for that stadium
 
  Transportation services
     I will need some hotel for my accommodation 
 
  Hotel Services 
  This service are expose to the world as APIs through internet, but How can we find those APIs?
  Simply answer for that question, is Google them one by one
  ( ;) I feel it is much easy to watch that cricket match in Television rather Goggling for each services. )
  How would you would you feel if you can find all your needed services (APIs) at one place. That is why we need APIs management. 
  API management refers to the process of publishing and managing APIs. 
  API management systems were first developed to use of Web APIs (likely Web services). Vendors of Internet applications, and Web 2.0 technologies and cloud computing services began publishing Web APIs. Widespread adoption of mobile devices was the major motivation for this. There was a growing need for integration with third-party applications built upon Web APIs.
  2006 The birth of API management as a software (Mashery). But it seems getting in to mainStream by now. There reason for me to say that is "For example, over 75% of Twitter and 60% of Salesforce usage is via APIs."
  I will give more on it with my next post. 
  
by Madhuka at April 01, 2012 03:46 PM

March 30, 2012


Heshan Suriyaarachchi—Monitor CPU Utilization of a Amazon EC2 instance using Amazon CloudWatch
Following code snippet show how to monitor CPU Utilization of a Amazon EC2 instance using Amazon Cloud Watch. In order to monitor the instance, Cloud Watch Monitoring should be enabled for the running instance.
private double monitorInstance(AWSCredentials credential, String instanceId) {
   try {
       AmazonCloudWatchClient cw = new AmazonCloudWatchClient(credential) ;

       long offsetInMilliseconds = 1000 * 60 * 60 * 24;
       GetMetricStatisticsRequest request = new GetMetricStatisticsRequest()
               .withStartTime(new Date(new Date().getTime() - offsetInMilliseconds))
               .withNamespace("AWS/EC2")
               .withPeriod(60 * 60)
               .withDimensions(new Dimension().withName("InstanceId").withValue(instanceId))
               .withMetricName("CPUUtilization")
               .withStatistics("Average", "Maximum")
               .withEndTime(new Date());
       GetMetricStatisticsResult getMetricStatisticsResult = cw.getMetricStatistics(request);

       double avgCPUUtilization = 0;
       List dataPoint = getMetricStatisticsResult.getDatapoints();
       for (Object aDataPoint : dataPoint) {
           Datapoint dp = (Datapoint) aDataPoint;
           avgCPUUtilization = dp.getAverage();
           log.info(instanceId + " instance's average CPU utilization : " + dp.getAverage());
       }

       return avgCPUUtilization;

   } catch (AmazonServiceException ase) {
       log.severe("Caught an AmazonServiceException, which means the request was made  "
               + "to Amazon EC2, but was rejected with an error response for some reason.");
       log.severe("Error Message:    " + ase.getMessage());
       log.severe("HTTP Status Code: " + ase.getStatusCode());
       log.severe("AWS Error Code:   " + ase.getErrorCode());
       log.severe("Error Type:       " + ase.getErrorType());
       log.severe("Request ID:       " + ase.getRequestId());

   }
   return 0;
}

by Heshan Suriyaarachchi at March 30, 2012 12:10 AM

March 29, 2012


Ishani Ratnayake—How to Deploy WSO2 ESB 3.2.x and above in Apache Tomcat
All WSO2 products are recommended to be deployed as stand alone servers and  we have stop supporting deploying on other 3rd party appservers from the latest releases.

In this blog post I will walk through on how you deploy Apache Tomcat 6.x.x with the new ESB which could be downloaded here. Also note that this blog will be valid from ESB 3.2.x.

Step 1
Extract the downloaded ESB to a desired location. The home directory will now be known as $ESB_HOME from here onwards.

Step 2
Extract Apache Tomcat 6.x.x to the required location which now will be known as $CATALINA_HOME. Downloadable here.

Step 3
Create a 'esb' folder within $CATALINA_HOME/webapps and copy WEB_INF folder which exists in $ESB_HOME/lib/core. You may also copy log4j.properties in $ESB_HOME/lib in to $CATALINA_HOME/webapps/esb/WEB-INF/classes to facilitate logging.

Step 4
Also copy repository and resources folders from $ESB_HOME and create a separate folder 'esb_repo' and copy them in.

From here onwards there requires some configuration tuning for them to work together.

Step 5
Edit $CATALINA_HOME/conf/server.xml to entertain https ports. Note that path of the esb_repo should be as for your location.

<connector port="8443" maxhttpheadersize="8192" maxthreads="150" minsparethreads="25" maxsparethreads="75" enablelookups="false" disableuploadtimeout="true" acceptcount="100" scheme="https" secure="true" sslenabled="true" clientauth="false" sslprotocol="TLS" keystorefile="/esb_repo/repository/resources/security/wso2carbon.jks" keystorepass="wso2carbon">

Step 6
Now we have to complete necessary changes in 'esb_repo' to start successfully
axis2.xml (esb_repo/repository/conf)

Go to the TransportListener and TransportSender section which uses org.apache.synapse.transport.nhttp.HttpCoreNIOSSLListener and change the keystore and truststore absolute paths as per accordingly. Remeber to change this in both TransportListener and TransportSender.

<parameter name="keystore" locked="false">
      </parameter></connector>
          /esb_repo/repository/resources/security/wso2carbon.jks<connector port="8443" maxhttpheadersize="8192" maxthreads="150" minsparethreads="25" maxsparethreads="75" enablelookups="false" disableuploadtimeout="true" acceptcount="100" scheme="https" secure="true" sslenabled="true" clientauth="false" sslprotocol="TLS" keystorefile="/esb_repo/repository/resources/security/wso2carbon.jks" keystorepass="wso2carbon"><parameter name="keystore" locked="false"><keystore><location></location>
          <type>JKS</type>
          <password>wso2carbon</password>
          <keypassword>wso2carbon</keypassword>
      </keystore>
  </parameter>
  <parameter name="truststore" locked="false">
      <truststore>
          <location>/esb_repo/repository/resources/security/client-truststore.jks</location>
          <type>JKS</type>
          <password>wso2carbon</password>
      </truststore>
  </parameter>

Also change the synapse_config directory absolute path accordingly.
<parameter name="SynapseConfig.ConfigurationFile" locked="false">/esb_repo/repository/deployment/server/synapse-configs</parameter>

Carbon.xml
You may find this file in the same location as per axis2.xml.
Change the Server_URL, WebContextRoot and RegistryHttpPort configurations as per follows

<serverurl>https://localhost:8443/esb/services/</serverurl>

<webcontextroot>/esb</webcontextroot>

<registryhttpport>8080</registryhttpport>

registry.xml
Change the necessary db url to absolute paths as per follows;
<url>jdbc:h2:/esb_repo/repository/database/WSO2CARBON_DB;DB_CLOSE_ON_EXIT=FALSE</url>

user-mgt.xml
Change the necessary db url property to absolute paths as per follows;
<property name="url">jdbc:h2:/esb_repo/repository/database/WSO2CARBON_DB;DB_CLOSE_ON_EXIT=FALSE</property>

Now we are done with the configurations :)

- Go ahead and declare the CARBON_HOME in a console;

export CARBON_HOME=/esb_repo

Start apache tomcat by cd'ng to $CATALINA_HOME/bin and using
./catalina.sh run

Upon starting the WSo2 ESB you can access the console with the server URL https://localhost:8443/esb/services/

Post in any comments if there are problems if you find any in deploying. Best of Luck!
by Ishani at March 29, 2012 04:44 PM

March 28, 2012


Supun Kamburugamuva—K-Means algorithm implementation in Java
We have given a homework assignment in Data-Mining class to implement the k-means algorithm  and it was a pretty exciting and fun home work assignment to work on.  We could get our hands dirty with a real data set and a real data mining  algorithm. So I thought of putting the work in github so that everyone can use the algorithm.



K-Means is a simple yet very powerful algorithm for clustering data. It  is a predictive algorithm for determine the clusters. There is a whole  lot of research done around K-Means because it provides a fast and  reliable solutions for most practical applications. The idea is to put  the data points in to a cluster with the smallest distance from the  clusters mean to the data point. We do this iteratively until we hit  some threshold. These threshold can be very simple like constant number  of iterations or it can be very complex based on the variance and the  quality of the data in clusters. 



The algorithm I have developed is pretty generic one and we've used the Wisconsin breast cancer data as the sample data for clustering.Bulk of the Code is pretty generic and can be used for clustering other data sets as well.



One of the best things about the k-means is that there are so many  parameters to tune it. This means depending on the nature of the data  set user has to do some investigations to pick the correct set of  parameters. In the given implementation I have made the following  parameters configurable. I have included the default implementation of  these configurations but user can extend the code to integrate their own  as well.



 Number of Centroids - This is the number of centroids to be used, and is the k of the algorithm



 Centroid Initialization - User has the choice of two method. 




Data is put in to k partitions randomly and the mean of these partitions are taken as the initial centroids
k data points are selected randomly and used as the centroids

Splitting - If a partition becomes empty we are going to split another partition  in to two so that we can maintain the k number of partitions. User has  the choice of selecting two different splitting methods.


Get the largest partition and split it in to two equal parts randomly
Get the partition with the largest variance and get the data point that is at a maximum distance from the mean of that partition

Distance algorithm - The default implementation has 3 built in algorithms for calculating the distances.


Euclidean distance
Euclidean squared distance
Manhattan distance



Stopping threshold - Here also user has two choices


Algorithm can be stopped at a constant number of iterations
Or algorithm is stopped when we get the same set of centroids  repeatedly. Here to compare the two centroids a comparison error can be  specified

At the end of clustering we calculated the PPV value of the clusters.  Also we did a 10 fold cross validation of the data as well.



The source code can be found in github and the url is https://github.com/supunkamburugamuva/Kmeans. The source is distributed  under Apache License Version 2 and you are welcome to contribute and use  the code without any restrictions.

by Supun Kamburugamuva at March 28, 2012 02:13 PM


Saliya Ekanayake—Blogging: Images in Comments
Finally, an awesome solution to a problem that I’ve been searching for quite a while: how to add an image in a comment to blog post?
 Look no further, just visit Spice Up Your Blog on this at http://www.spiceupyourblog.com/2010/12/images-colored-text-blogger-comments.html
 Apparently it has just more than adding images like colored text and scrolling text.
 
See the test comments I made for fun
  
by Saliya Ekanayake at March 28, 2012 06:17 AM


Saliya Ekanayake—Windows Live Writer: Life Made Easy for Blogging
I’ve been using Blogger for quite some time and over the years they have improved their Web based blog editor quite a lot, yet there was some uneasiness always when thinking about editing or adding a post. Anyway, after I got to know about Windows Live Writer I wanted to get it setup with Blogger and since today it wasn’t successful simply for some reason I wouldn’t understand “Username or password is incorrect”! I remembered only today that I was using two step verification with Google and that I have to create an application specific password to connect to Blogger. Wish it came to my mind sooner! Anyway, it’s now working fine and if you can read this online that means I made a post successfully with Live Writer.
 Few places to note if you are having trouble connecting to Blogger with Live Writer as I did.
  Blog URL: Don’t forget to use https instead of http
 Username: Remember to add @gmail.com to your user id
 Password: As mentioned above, if you are using a two step verification with Google you need to generate application specific password to connect (see http://support.google.com/accounts/bin/answer.py?hl=en&answer=185833)
  
by Saliya Ekanayake at March 28, 2012 04:45 AM

March 27, 2012


Saliya Ekanayake—SugarSync Public Link: Direct Download
SugarSync public links are great, but unfortunately they've added an intermediate download page instead of the direct download that used to be there (see more on this at http://www.sugarsync.com/blog/2012/01/10/if-you-love-your-public-links-set-them-free/). 

May be it was done with good intentions, but it broke all the image links we had in our Web site. Anyway, it seems there's a way around to get images working back in your Web pages without much hassle. 

The solution is just add the following to the end of each image link (I know it's bit work too, but way better than having copy images to a local folder and linking them again manually).

old-link?directDownload=true Update 3/27/2012: 
 I tried the same trick to put an image to a blog post using a SugarSync public link, but it wasn’t successful. As it seems Blogger’s image retrieving service couldn’t handle the directDownload=true.
  
by Saliya Ekanayake at March 27, 2012 11:04 PM


Kathiravelu Pradeeban—GSoC Feedback Session in IRC for the rejected organizations
Google schedules an irc feedback session with the organizations that couldn't make to the summer each year, addressing what went wrong with the each organizations. It was an interesting session today. 


First we all adjust our irc nick to have the format {org}|{nick}
/nick ogsadai|Pradeeba


Then we queue ourselves with the gsoc bot by sending a message in the following format.
/msg gsocbot queue  

/msg gsocbot queue ogsadai http://sourceforge.net/apps/trac/ogsa-dai/wiki/GoogleSummerOfCode



To see the next 10 positions,

/msg gsocbot showqueue

(11:17:59 PM) gsocbot: 01. AmineKhaldi reactos http://reactos.org/wiki/Google_Summer_of_Code_2012_Ideas

(11:18:00 PM) gsocbot: 02. ReactOS|Amine reactos http://reactos.org/wiki/Google_Summer_of_Code_2012_Ideas

(11:18:01 PM) gsocbot: 03. mikhas maliit https://wiki.maliit.org/Ideas

(11:18:02 PM) gsocbot: 04. prism PRISM Model Checker http://www.prismmodelchecker.org/gsoc/

(11:18:03 PM) gsocbot: 05. ogsadai|Pradeeba ogsadai http://sourceforge.net/apps/trac/ogsa-dai/wiki/GoogleSummerOfCode

(11:18:04 PM) gsocbot: 06. Andrew__ boost https://svn.boost.org/trac/boost/wiki/SoC2012

(11:18:05 PM) gsocbot: 07. plan9|anth plan9 http://www.plan9.bell-labs.com/wiki/plan9/gsoc-2012-ideas/index.html

(11:18:06 PM) gsocbot: 08. plone|kevin7kal plone http://plone.org/events/summer-of-code/2012/

(11:18:07 PM) gsocbot: 09. wordpress|jane WordPress http://codex.wordpress.org/index.php?title=GSoC2012&oldid=114656

(11:18:08 PM) gsocbot: 10. tcl_aku tcl http://wiki.tcl.tk/28789

(11:18:09 PM) gsocbot: ... and more ...



This year OGSA-DAI applied as a mentoring organization. Carol mentioned that the organization's application as well as the idea's page look perfect, while suggesting we could have proposed more ideas. OGSA-DAI will come up with more ideas next year. In the mean time, feel free to volunteer for OGSA-DAI, if you are interested. OGSA-DAI community always welcomes the enthusiastic volunteers.

by Kathiravelu Pradeeban at March 27, 2012 04:59 PM


Supun Kamburugamuva—Password protect a file in Linux
We can use the gpg utility to password protect a file in Unix based systems. The command



gpg -c filename



will ask for a password and create a new file with the extention .gpg. Now the new file is a password protected file.



To decrypt the file, just use



gpg "protectedfilename"



It will ask for the password and decrypt the file.

by Supun Kamburugamuva at March 27, 2012 03:18 PM

Footnotes


Info
Last updated:
May 21, 2012 06:01 PM
Powered by:
Export:


Subscriptions

 Afkham AzeezDead Letter Channel EIP in WSO2 ESB: An example
Winners vs. Losers
Short Summary of HTTP Status Codes
Scatter-Gather EIP with WSO2 ESB
Dead Letter Channel EIP with WSO ESB
Lazy Loading Webapps in a PaaS Deployment
The role of a Load Balancer in a Platform-as-a-Service
Is PaaS on top of IaaS the proper way to go?
This blog template uses SyntaxHighlighter hosted on StratosLive
How to setup WSO2 Elastic Load Balancer
My Top 10 Third Party IntelliJ IDEA Plugins
Lazy Loading Deployment Artifacts in a PaaS Deployment
WSO2 ESB - Service Chaining
The birth of Stratos multi-tenancy
6 years at WSO2!
WSO2 Load Balancer - how it works
 Ajith Ranabahu
 Amila MaharachchiGetting ActiveMQ Redelivery to work with WSO2 ESB
 Buddhika ChamithSome Hadoop and Hive Gotchas and Developer Tips
Learn by Errors : Java + OSGi
 Chamara AriyarathneOpenLDAP Clustering Guide
 Chanaka Jayasena
 Charith Dhanushka WickramarachchiAnother message redelivery pattern with WSO2 ESB
Distributed Coordination with Apache Zookeeper - A Java Approch
 Charitha KankanamgeManaging SOA artifacts in different environments using WSO2 Governance Registry
 Chathura Ekanayake
 Chathuri Wimalasena
 Chintana Wilamuna
 Chris HaddadContent Gets More Personal with the Cloud
Re-invent Software Delivery and Offer Your Business as a Service
PaaS TCO and PaaS ROI: Multi-tenant, shared container PaaS
Progress Sonic to Exit Middleware Market
VMWare’s Linux in the Cloud Strategy
Know your Cloud Dimensions
How to pick an ESB? An Enterprise Service Bus Evaluation Framework
What is WSO2 AppFactory?
Pinterest API and the Money Board
An Enhanced User Experience with the WSO2 Mobile Services Gateway
SCIM Simple Cloud Identity Management Interop
 Dakshitha RatnayakeLogging with WSO2 ESB
 Damitha Kumarage
 Danushka Menikkumbura
 Deepal Jayasinghe
 Deependra Ariyadewa
 Denis WeerasiriIntroduction and Advanced Concepts of BPEL
Business Process Hosting in the Cloud
Enabling logging for various components in WSO2 BPS
WS-BPEL 2.0 Extension Activity Development for WSO2 BPS
Writing a BPEL compensation handler
Wishing you a very happy new year 2012!
Development and Deployment Best Practices for WSO2 BPS
 Devaka Randeniya
 Dhananjaya jayasinghe
 Dimuthu Gamage
 Eran Chinthaka
 Evanthika AmarasiriHow to resolve 'ORA-01882: timezone region not found' when starting WSO2 product servers with Oracle
 Ganesh PrasadTight Coupling in the TCP/IP Stack!
The Unholy Alliance of Analyst and Big Vendor
Mentoring NoSQL From Adolescence to Maturity
An Optimistic Approach to Identity
Dimensions of Decoupling
 Gillian Dass
 Harshana Eranga Martin
 Hasini GunasingheSetting up cloud manager of Stratos with an external LDAP
Notes from IETF 83rd Meeting
SCIM Interop Event at IETF 83rd Meeting
 Heshan SuriyaarachchiMonitor CPU Utilization of a Amazon EC2 instance using Amazon CloudWatch
 Hiranya JayathilakaWSO2 ESB Tips & Tricks 09: Introduction to REST APIs
 Ishani RatnayakeWho is an Agile BA ?
How to Deploy WSO2 ESB 3.2.x and above in Apache Tomcat
 Isuru Suriarachchi4 years at WSO2
Apache CXF support in WSO2 Application Server
 Kalani Ruwanpathirana
 Kasun GunathilakeHow to remote debug Apache Cassandra standalone server
 Kathiravelu PradeebanDeveloping BPEL Processes using WSO2 Carbon Studio
All Stuff Cloudyy - WSO2 Stratos
Moving from a 'Platform' to the 'Platform-as-a-Service' ~ What is it all about?
"Reading Level" ~ Google gets more interesting!
[IJCISIM] Horizontal Format Data Mining with Extended Bitmaps
Digging into addictions
Apache Meetup, Kandy and Community Matters!
Google Summer of Code 2012 and AbiWord
DZone Kolamba Meetup
PART-2
Incubators!
The GSoC community activities in Sri Lanka
How Auto Scaling works with WSO2 Load Balancer
StratosLive - A case study for WSO2 Load Balancer
GSoC Feedback Session in IRC for the rejected organizations
 Keith Chapman
 Krishantha SamaraweeraSelenium WebDriver : Handling mouseover events
 Lahiru Sandakith
 Madhuka UdanthaJUnit Test for Beginners
Clean Your Code with Sonar
Installing Sonar and PMD plugins for Eclipse
What is API management? (part 1)
 Manisha Eleperuma
 Manjula Peiris
 Muhammed Shariq
 Nanadana Mihindukulasooriya
 Nandika Jayawardana
 Nuwan BandaraBandos’ Arcade, gets a new look and feel
 Paul Fremantle
 Prabath AbeysekeraRetrieve the classes implementing a particular interface programmatically in Java
Query UDTs(User Defined Types) with WSO2 Data Service Server
 Prabath SiriwardenaRunning two OpenLDAP instances in the same machine under MAC OS X
Setting up OpenLDAP under MAC OS X
OWASP Sri Lankan chapter inaugural meeting...
 Prabath Siriwardena
 RampartFAQ
 Ranga Siriwardena
 Ruchith FernandoMy Poster on Private Anonymous Messaging
Gmail and Spam
 Ruwan Janapriya
 Sagara GunathungaRelease of Axis2 1.6.2 and  Sandesha2 1.6.2 , Rampart 1.6.2
Code generation and WSDL exposed over HTTPS
How easy to test your web service over HTTPS
Axis2 clustering on Tomcat
Axis2 JMS transport and ActiveMQ
How to use Axis2 WSDL2JAVA with proxy authentication
 Saliya EkanayakeRecursion is Natural
Taming Wild Horses: Chapel Asynchronous Tasks
Chapel is Sweet
Download a Set of URLs with GNU Wget
Blogging: Images in Comments
Windows Live Writer: Life Made Easy for Blogging
SugarSync Public Link: Direct Download
 Sameera Jayasoma
 Saminda Abeyruwan
 Samisa Abeysinghe
 Sanjeewa MalalgodaTenant Aware Load balancer
How to avoid loading error in soapui on ubuntu 11.04 (ERROR [SoapUI] An error occured [no protocol: POECommonEPrescribeService-wsdl-soapui-project.xml])
Tenant Aware Load Balancer Concept
Carbon Context and it's usages in WSO2 Products
 Sanjiva Weerawarana
 Sanka Samaranayake
 Senaka Fernando
 Srinath PereraHow to scale Complex Event Processing (CEP) Systems?
How to a measure the Performance of a Server?
Introduction to Open Source, Apache and Apache Way
Scaling Distributed Queues: A Short Survey
Authentication and Authorization Choices in WSO2 Platform
Generating a Distributed Sequence Number
 Supun KamburugamuvaCounting the code lines in a Java program
K-Means algorithm implementation in Java
Password protect a file in Linux
 Tharindu MathewUseful commands for debugging OSGI issues and constraints
Solving the XML Parsing error in Firebug
Do you trust Google Big Query with your Big Data?
Using Google collections’ MapMaker to quickly build a cache
 Thilina Buddhika
 Thilina Gunarathne
 Thilini Ishaka
 Waruna RanasingheWebinar - Using WSO2 Carbon to Help Convert One-Sixth of New Zealand’s Population into Mobile Telecom Customers
 Yumani RanaweeraHierarchical service deployment support in WSO2 AS
Cheat-sheet : Starting WSO2-AS-4.2.1 with Activemq-5.2.2

On-Premise PaaS vs. On-Premise Hosting	Shared Container PaaS Scenario	Traditional Deployment Template	Traditional – Scenario 1	Traditional – Scenario 2	Traditional – Scenario 3	Traditional – Scenario 4
Tenants and Partitioning
Tenant Isolation Level	Shared PaaS Nodes	Dedicated Instances	Dedicated Instances	Dedicated Instances	Dedicated Instances	Dedicated Instances
Tenants	100	1	15	16	50	100
PaaS Management Services [# of JVMs]	11	0	0	0	0	0
PaaS Controller and Registry	3	0	0	0	0	0
Load Balancer for management traffic	3	0	0	0	0	0
Cloud Manager (provisioning, metering, billing)	1	0	0	0	0	0
Business Activity Monitor and summarizer	4	0	0	0	0	0
Application Platform Services [# of JVMs]
Application Server	14	2	30	32	100	200
Gadget Server	0	0	0	0	0	0
Mashup Server	0	0	0	0	0	0
Enterprise Service Bus	14	2	30	32	100	200
Governance Registry	2	1	15	16	50	100
Identity Server	2	1	15	16	50	100
Business Process Management	0	0	0	0	0	0
Business Rules	0	0	0	0	0	0
Complex Event Processing Server	0	0	0	0	0	0
Data Services Server	4	1	15	16	50	100
Message Broker	0	0	0	0	0	0
Cassandra Storage Service	0	0	0	0	0	0
Relational Storage Service	0	0	0	0	0	0
File Storage Service	0	0	0	0	0	0
API Management	0	0	0	0	0	0
Business Activity Monitoring	0	0	0	0	0	0
Load Balancer for Application Platform Service Clusters	3	0	0	0	0	0
Platform Footprint and Cost
Number of JVM Instances	50	7	105	112	350	700
Cost Per JVM Instance	$8,000	$8,000	$8,000	$8,000	$8,000	$8,000
Total Platform Subscription Cost	$400,000	$56,000	$840,000	$896,000	$2,800,000	$5,600,000
Platform Subscription Cost Comparison (App Platform)	$0	($344,000)	$440,000	$496,000	$2,400,000	$5,544,000
Infrastructure as a Service
JVM Density per IaaS node (32GB IaaS, 4GB JVM)	8	8	8	8	8	8
Calculated number of IaaS nodes	7	1	14	14	44	88
Cost per IaaS Year	$7,908	$7,908	$7,908	$7,908	$7,908	$7,908
Total IaaS Cost	$55,356	$7,908	$110,712	$110,712	$347,952	$695,904
Platform Subscription Cost Comparison (IaaS)	$0	($47,448)	$55,356	$55,356	$292,596	$640,548
IT Administrative and Management Cost
Chargeback/showback billing	12000	60	900	960	3000	6000
Tenant configuration	610	60	870	900	2820	5640
Tenant monitoring and systems administration	2600	364	5460	5824	18200	36400
Cost per IT man hour	$60	$60	$60	$60	$60	$60
WSO2 Stratos Platform Training Cost	$25,000	$0	$0	$0	$0	$0
Total IT Administrative and Management Cost	$937,600	$29,040	$433,800	$461,040	$1,441,200	$2,882,400
Platform Subscription Cost Comparison (IT Admin & Mgt)	$0	($908,560)	($503,800)	($476,560)	$503,600	$1,944,800
Total Cost of Ownership
Total Cost	$1,392,956	$92,948	$1,384,512	$1,467,752	$4,589,152	$9,178,304
Total Savings / (Extra Expense)	$0	($1,300,008)	($8,444)	$74,796	$3,196,196	$7,785,348
JVM Instance Efficiency		0.1	2.1	2.2	7.0	14.0
Cost Efficiency		7%	99%	105%	329%	659%

On-Premise Shared PaaS vs. On-Premise Single Tenant Hosting	Shared Container PaaS Scenario	Traditional Deployment Template	Traditional – Scenario 1	Traditional – Scenario 2	Traditional – Scenario 3	Traditional – Scenario 4
Tenants and Partitioning
Tenant Isolation Level	Shared PaaS Nodes	Dedicated Instances	Dedicated Instances	Dedicated Instances	Dedicated Instances	Dedicated Instances
Tenants	8	1	2	4	6	8
PaaS Management Services [# of JVMs]	8	0	0	0	0	0
PaaS Controller and Registry	3	0	0	0	0	0
Load Balancer for management traffic	2	0	0	0	0	0
Cloud Manager (provisioning, metering, billing)	1	0	0	0	0	0
Business Activity Monitor and summarizer	2	0	0	0	0	0
Application Platform Services [# of JVMs]
Application Server	2	2	4	8	12	16
Gadget Server	0	0	0	0	0	0
Mashup Server	0	0	0	0	0	0
Enterprise Service Bus	2	2	4	8	12	16
Governance Registry	2	1	2	4	6	8
Identity Server	1	1	2	4	6	8
Business Process Management	0	0	0	0	0	0
Business Rules	0	0	0	0	0	0
Complex Event Processing Server	0	0	0	0	0	0
Data Services Server	1	1	2	4	6	8
Message Broker	0	0	0	0	0	0
Cassandra Storage Service	0	0	0	0	0	0
Relational Storage Service	0	0	0	0	0	0
File Storage Service	0	0	0	0	0	0
API Management	0	0	0	0	0	0
Business Activity Monitoring	0	0	0	0	0	0
Load Balancer for Application Platform Service Clusters	2	0	0	0	0	0
Platform Footprint and Cost
Number of JVM Instances	18	7	14	28	42	56
Subscription cost per JVM Instance [per year]	$8,000	$8,000	$8,000	$8,000	$8,000	$8,000
Total Platform Subscription Cost [per year]	$144,000	$56,000	$112,000	$224,000	$336,000	$448,000
Platform Subscription Cost Comparison (App Platform)	$0	($88,000)	($32,000)	$80,000	$192,000	$392,000
Infrastructure as a Service
JVM Density per IaaS node (32GB IaaS, 4GB JVM)	8	8	8	8	8	8
Calculated number of IaaS nodes	3	1	2	4	6	7
Cost per IaaS node [per year]	$7,908	$7,908	$7,908	$7,908	$7,908	$7,908
Total IaaS Cost [per year]	$23,724	$7,908	$15,816	$31,632	$47,448	$55,356
Platform Subscription Cost Comparison (IaaS)	$0	($15,816)	($7,908)	$7,908	$23,724	$31,632
IT Administrative and Management Cost
Chargeback/showback billing	960	60	120	240	360	480
Tenant configuration	178	60	120	240	360	450
Tenant monitoring and systems administration	936	364	728	1456	2184	2912
Cost per IT man hour	$60	$60	$60	$60	$60	$60
WSO2 Stratos Platform Training Cost	$25,000	$0	$0	$0	$0	$0
Total IT Administrative and Management Cost [per year]	$149,440	$29,040	$58,080	$116,160	$174,240	$230,520
Platform Subscription Cost Comparison (IT Admin & Mgt)	$0	($120,400)	($91,360)	($33,280)	$24,800	$81,080
Total Cost of Ownership
Total Cost [Year 1]	$317,164	$92,948	$185,896	$371,792	$557,688	$733,876
Total Savings for PaaS / (Extra Expense)	$0	($224,216)	($131,268)	$54,628	$240,524	$416,712
JVM Instance Efficiency		0.4	0.8	1.6	2.3	3.1
Cost Efficiency		29%	59%	117%	176%	231%

Topology	Pros	Cons	Supporting Systems
Master Slave	Support HA	No Scalability	Qpid, ActiveMQ, RabbitMQ
Queue Distribution	Scale to large number of Queues	Does not scale for large number of messages for a queue	RabbitMQ
Cluster Connections	Support HA	Might not support in-order delivery Logic runs in the client side takes local decisions.	HorentMQ
Broker/Queue Networks	Load balancing and distribution	Fair load balancing is hard	ActiveMQ

Cloud terms	sharing	location	responsibility
public-external-outsourced	3	3	1
community-external-outsourced	2	3	2
private-external-outsourced	1	3	1
private-external-on-premise	1	3	2
public-internal-on-premise	3	1	3
community-internal-on-premise	2	1	3
private-internal-on-premise	1	1	3
private-internal-outsourced	1	1	2
Your requirements	?	?	?

WSO2 Venus

May 21, 2012

Kathiravelu Pradeeban—Developing BPEL Processes using WSO2 Carbon Studio

Buddhika Chamith—Some Hadoop and Hive Gotchas and Developer Tips

localhost: Error: JAVA_HOME is not set

Name node mysteriously fails to start

java.io.IOException Call to localhost/127.0.0.1:9000 failed on local exception: java.io.EOFException

Debugging Hadoop Local (Standalone) mode

Debugging Hive Server

Sanjeewa Malalgoda—Tenant Aware Load balancer

Sanjeewa Malalgoda—How to avoid loading error in soapui on ubuntu 11.04 (ERROR [SoapUI] An error occured [no protocol: POECommonEPrescribeService-wsdl-soapui-project.xml])

Krishantha Samaraweera—Selenium WebDriver : Handling mouseover events

Chris Haddad—Content Gets More Personal with the Cloud

May 20, 2012

Kathiravelu Pradeeban—All Stuff Cloudyy - WSO2 Stratos

Amila Maharachchi—Getting ActiveMQ Redelivery to work with WSO2 ESB

Afkham Azeez—Dead Letter Channel EIP in WSO2 ESB: An example

Afkham Azeez—Winners vs. Losers

Afkham Azeez—Short Summary of HTTP Status Codes

May 19, 2012

Charith Dhanushka Wickramarachchi—Another message redelivery pattern with WSO2 ESB

Charitha Kankanamge—Managing SOA artifacts in different environments using WSO2 Governance Registry

May 18, 2012

Supun Kamburugamuva—Counting the code lines in a Java program

Srinath Perera—How to scale Complex Event Processing (CEP) Systems?

What is CEP?

What is Scaling CEP?

How to Scale CEP?

Handling Large Number of Queries

Queries that need large working memory

Handling large number of events handling a complex query that might not fit within a single machine

Srinath Perera—How to a measure the Performance of a Server?

Understanding Server Performance

Doing the Performance Test

Workload and client Setup

Experimental Setup

Running the test

Verifying your results

Analyze your results and write it up

Tharindu Mathew—Useful commands for debugging OSGI issues and constraints

1. ss / ss <string> (abbr. for short status)

2. b <id> (abbr. for bundle)

3. p <package> (abbr. for packages)

4. diag <bid> (abbr. for diagnose)

5. ls (abbr. for list services)

6. comp <component id> or ls -c <bundleid>

Evanthika Amarasiri—How to resolve 'ORA-01882: timezone region not found' when starting WSO2 product servers with Oracle

May 17, 2012

Nuwan Bandara—Bandos’ Arcade, gets a new look and feel

May 16, 2012

Tharindu Mathew—Solving the XML Parsing error in Firebug

Afkham Azeez—Scatter-Gather EIP with WSO2 ESB

Underneath the Hood

Chris Haddad—Re-invent Software Delivery and Offer Your Business as a Service

Re-invent software delivery

Step 1. Identify core business capabilities and outsource commodity

Step 2. Iteratively build extensible, configurable business services, APIs, and applications

Step 3. Increase business partnerships with API management’s on-demand self-service and analytics

Step 4. Host services, APIs and applications on a multi-tenant platform and facilitate per-tenant configuration

Summary

May 15, 2012

Afkham Azeez—Dead Letter Channel EIP with WSO ESB

Buddhika Chamith—Learn by Errors : Java + OSGi

java.lang.VerifyError

java.lang.IncompatibleClassChangeError

java.lang.LinkageError : loader constraint violation in xxxx – blah …

Package uses conflict: Import-Package: yyy; version=”x.x.x”

java.lang.UnsatisfiedLinkError

May 14, 2012

Kathiravelu Pradeeban—Moving from a 'Platform' to the 'Platform-as-a-Service' ~ What is it all about?

Saliya Ekanayake—Recursion is Natural

Example 1: Is Even?

Example 2: Factorial

Example 2: Length of a List

Example 3: Map f() to List S

Waruna Ranasinghe—Webinar - Using WSO2 Carbon to Help Convert One-Sixth of New Zealand’s Population into Mobile Telecom Customers

May 13, 2012

Saliya Ekanayake—Taming Wild Horses: Chapel Asynchronous Tasks

Chris Haddad—PaaS TCO and PaaS ROI: Multi-tenant, shared container PaaS

Cost Calculation Methodology

`2. b <id> (abbr. for bundle)`