How to Make WSO2 Elastic Load Balancer highly available through AWS Elastic Load Balancer

Table of contents

• Prerequisites
• Introduction
• Applies to
• Getting Started
• Configuring WSO2 Services
• Configuring Amazon ELB
• Summary

Pre-requisites

• WSO2 ELB
  • https://wso2.com/products/elastic-load-balancer
• WSO2 Application Server
  • https://wso2.com/products/application-server
• JDK
  • https://www.oracle.com/technetwork/java/javase/downloads/index.html
• AWS Account
  • https://aws.amazon.com

Introduction

High availability in a system can be achieved at deferent levels. WSO2 ELB makes WSO2 services highly available even in the event of a failure. Nevertheless, WSO2 ELB itself can be a single point of failure. Therefore, making WSO2 ELB highly available is crucial for a system. There are several ways to achieve this. This article explains how to achieve high availability for WSO2 ELB using a few free and open-source tools and packages.

• https://docs.wso2.org/display/ELB203/Automatic+Failover

If we run WSO2 services in a Amazon EC2 environment, we can achieve high availability for WSO2 ELB using Amazon Elastic Load Balancing service. In this tutorial, we will be setting up an WSO2 Application Server fronted by two WSO2 ELB instances.

Applies to

For this tutorial, we use a one Amazon ELB, two WSO2 ELB instances, and two WSO2 Appserver instances.

Getting started

Deployment architecture

1. Start 4 Amazon EC2 instances. (m1.medium instances would be sufficient)
2. Install JDK and set JAVA_HOME environment variable pointing to JDK installation directory in all 4 instances.

We will be using the following hostnames ELBs so it will be easier to understand.

• ELB 1 - elb01.ec2-test.wso2.com
• ELB 2 - elb02.ec2-test.wso2.com

Configuring WSO2 services

You have to configure a WSO2 Application Server cluster as well as two WSO2 ELB instances to front the Application Server cluster. We will be configuring the simplest Appserver cluster to minimize the configurations. We have removed comments from configuration files for brevity. Please refer to this document to read more about clustering. Only the values that need to be changed in configuration files have been included.

WSO2 Application Server

Carry out the following changes in both application servers.

Clustering configuration in axis2.xml

    <clustering enable="true" class="org.apache.axis2.clustering.tribes.TribesClusteringAgent">
        <parameter name="membershipScheme">wka</parameter>
        <parameter name="domain">wso2.appserver.domain</parameter>
        <parameter name="localMemberBindAddress">local.ec2-test.wso2.com</parameter>
        <parameter name="localMemberHost">local.ec2-test.wso2.com</parameter>
        <parameter name="properties">
            <property value="https://${hostName}:${httpsPort}/services/" name="backendServerURL"></property>
            <property value="https://${hostName}:${httpsPort}/" name="mgtConsoleURL"></property>
            <property value="mgt" name="subDomain"/>
        </parameter>
        <members>
            <member>
                <hostname>elb01.ec2-test.wso2.com</hostname>
                <port>4000</port>
            </member>
            <member>
                <hostname>elb02.ec2-test.wso2.com</hostname>
                <port>4000</port>
           </member>
        </members>
    </clustering>

carbon.xml configuration

    <HostName>management.appserver.ec2-test.wso2.com</HostName>
    <MgtHostName>management.appserver.ec2-test.wso2.com</MgtHostName>

catalina-server.xml configuration

    <Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
        proxyPort="80" />

    <Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
        proxyPort="443" />

WSO2 ELB

Carry out the following changes in both WSO2 ELBs.

loadbalancer.conf configuration

    appserver {
        domains {
            wso2.appserver.domain {
                min_app_instances       1;
                hosts           management.appserver.ec2-test.wso2.com;
                sub_domain      mgt;
                tenant_range    *;
            }
        }
    }

axis2.xml configuration

This configuration is only added for easier access of the URL. The transport ports have been changed to default http/https ports so you can access the management console without typing the port.

e.g. www.ec2test.wso2.com instead of www.ec2test.wso2.com:9443

    <transportReceiver name="http" class="org.apache.synapse.transport.passthru.PassThroughHttpListener">
        <parameter name="port">80</parameter>
    </transportReceiver>
    <transportReceiver name="https" class="org.apache.synapse.transport.passthru.PassThroughHttpSSLListener">
        <parameter name="port" locked="false">443</parameter>
    </transportReceiver>

Now you have configured both WSO2 Appserver and WSO2 ELB. Go to each instance and start them

cd <path to service>/bin
./wso2server.sh start

check <path to service>/repository/logs/wso2carbon.log to check if there are backend logs. Also do the following to verify the Appserver cluster is working properly.

• Get the public IP of one of the WSO2 ELB instances. Ping the public DNS of the instance, then you should be able to see the IP of the instance
• Add the following host entry to your local machine

        <Public IP of the WSO2 ELB> management.appserver.ec2-test.wso2.com
    

You should be able to access Appserver by typing https://management.appserver.ec2-test.wso2.com in your browser

Configuring Amazon ELB

Still, we are half way through, as only the Appserver cluster is highly available. Now we can start configuring Amazon ELB to load balance between two WSO2 ELBs

• Login to the AWS management console and select "EC2" from the services list
• Click on the "Load Balancers" link on the left hand side and click on the "Launch Instance" button

Give the Load Balancer a meaningful name. In the "Listener Configuration:" section the standard HTTP port is already there by default. Add the standard HTTPS port as well. Make sure both "Load Balancer Port" and "Instance Port" are 443 when you are adding the HTTPS port.

Hit continue. Since we have selected the HTTPS port, we have to upload a SSL certificate. We have to extract the "Private Key" and "Public Key Certificate" from wso2carbon.jks, the default keystore provided by WSO2 services. You can find the wso2carbon.jks from repository/resources/security directory in any of the WSO2 service.

Extract Public/Private keys from wso2carbon.jks

Type the password "wso2carbon" each time it prompts

• Extract PK in PKCS12 format

keytool -v -importkeystore -srckeystore wso2carbon.jks -srcalias wso2carbon -destkeystore myp12file.p12 -deststoretype PKCS12

openssl pkcs12 -in myp12file.p12 -nocerts -nodes

openssl rsa -in private.pem -out server.key

server.key is the private key.

keytool -export -alias wso2carbon -keystore wso2carbon.jks -rfc -file public.cert

public.cert is the public key. Now copy and paste private/public keys in the AWS management console. Please note that you have to copy including "-----BEGIN RSA PRIVATE KEY-----" and "-----END RSA PRIVATE KEY-----". Give a meaningful name to cert.

Keep the default cipher policy that has been already selected in the next window.

In next window, select "Proceed without backend authentication" and continue

In next window, you can configure a health check so it will periodically check if the backend instance is up and running. This can be configured in various levels. In this tutorial, we will configure Amazon ELB to check if the port 80 is up in the backend instances. Configure the health check and continue.

In next window, you have to select the ec2 instances that we are going to load balance. We have configured two WSO2 ELB Instances, which include "EC2-test-elb-1" and EC2-test-elb2", so selecting those two instances.

Now everything is configured to launch a load balancer. In next window, review your configurations and press "create".

We have now completed creating the Amazon ELB; however, we still we have one more step before accessing the Management Console of Appserver. We need to enable "Stickiness" in AWS ELB level. In WSO2 services, sticky sessions are identified from the value of the JSESSIONID cookie. Therefore, we have to enable "Application generated Cookie Stickiness" in AWS ELB.

• Select the created ELB from AWS Management console. In this case it is "ec2-test"
• In "Port Configuration:" you will see that both port 80 and 443 are listed. First click "edit" on port 80 configurations.



• Select "Enable Application Generated Cookie Stickiness", put "JSESSIONID" as Cookie Name and save.



• Follow the same procedure to configure stickiness for port 443.

We have now completed the AWS ELB configurations. We have configured Application Server's host name as "management.appserver.ec2-test.wso2.com"s so we have to point that to AWS ELB as it's the top most ELB. The recommended way to do this is to get the DNS name of the AWS ELB instance and configure a CNAME pointing to the DNS name. The reason is that AWS ELB itself is highly available. If you ping the DNS name of the AWS ELB you will notice varied IP addresses from time to time. If you use only one IP address you will loose the HA in the AWS ELB layer. Since this is just a demo, we will get only one IP and add a host entry in my local machine pointing to that particular IP.

• <IP of the AWS ELB Instance> management.appserver.ec2-test.wso2.com

You can access the management console of WSO2 Application Server typing https://management.appserver.ec2-test.wso2.com in the browser. Try stopping one WSO2 ELB instance to verify if it works properly.

Summary

When it comes to WSO2 service deployments in a AWS EC2 environment, AWS ELB is a very useful service to achieve HA. However, you might still have to use WSO2 ELB as well because AWS ELB does not support some important features for WSO2 services such as service aware load balancing.