[Tutorial] How To Use WSO2 Puppet Modules to Deploy WSO2 Products
- Suhan Dharmasuriya
- Associate Technical Lead - WSO2
Table of contents
- Introduction
- Instructions
- FAQ
- Puppet master/agent communication errors
- How can I view the wso2carbon.log?
- How can I change the JDK version?
- How can I change JVM parameters?
- Conclusion
- References
Introduction
Using WSO2 Puppet Modules, this article will provide a sequence for you to follow in testing your WSO2 product of interest for a standalone deployment setup in a given puppet agent instance. This can be considered as a guide in setting up your own puppet master/agent environment given that you manually set up the VM instances.
We have separated data and deployment logic. Deployment specific information will be stored as Hieradata whereas puppet deployment logic will be kept separately inside puppet module manifests for a given WSO2 product puppet module. As you can see here for each WSO2 product there is a puppet module. All these puppet modules are dependent on the wso2base puppet module that contains all common required logic for all WSO2 products.
Once you set-up puppet master and puppet agent, you can indicate the product, product profile, product version, and other relevant information as factors in the puppet agent and it will set up the selected product for you with default settings. Detailed instructions will follow in the next section:
e.g.
product_name=wso2am product_version=1.10.0 product_profile=default vm_type=openstack
Please note that the structure of WSO2 Puppet Modules may change as it evolves over time, yet the concept will remain the same.
Instructions
For ease of understanding we have distributed the instructions under 3 Tasks.
- Task 1. Set up two instances
- Task 2. Set up puppet master and puppet agent
- Task 3. Perform a catalog run to set up product instance
Task 1 - Set up two instances
This can be either two cloud instances, two physical computer instances or two local VM instances. e.g. Two VM Instances of 2 VCPU, 4GB RAM with Ubuntu 14.04. You must be able to SSH to both the instances given that you have connectivity. One will be prepared as puppet master and the other as puppet agent.
Task 2 - Set up puppet master and puppet agent
In this task, first we will install the puppet master and the puppet agent. Then we will configure both master and agent, respectively.
2.1. Installing puppet
- Login to puppet master as the super user.
- Issue the following commands.
> ntpdate pool.ntp.org ; apt-get update && sudo apt-get -y install ntp ; service ntp restart > cd /tmp > wget https://apt.puppetlabs.com/puppetlabs-release-trusty.deb > dpkg -i puppetlabs-release-trusty.deb > apt-get update > apt-get install puppetmaster
- Check the puppet version as:
puppet -V
.Puppet version Should be 3.8.3 or higher.
- Edit
etc/hostname
file and set property hostname.puppetmaster
- Edit your puppet master hosts file
/etc/hosts
as follows and set puppet master hostname.127.0.0.1 localhost
127.0.0.1 puppetmaster
- Login to puppet agent as the super user.
- Issue the following commands.
> apt-get update > apt-get install puppet
- Edit
etc/hostname
file and set property hostname.qaa-node-1
- Edit /etc/hosts file of your puppet agent as follows. Puppet agent will use this to find the relevant puppet master’s IP address.
127.0.0.1 localhost
192.168.19.XXX puppet # puppet master's IP address
2.2. Configuring puppet
- Login to puppet master as the super user.
- Create a directory
/etc/puppet/environments/production/
.
Note: To learn more about puppet environments you can refer this.
- Modify
/etc/puppet/puppet.conf
file of puppet master appending the following to [main]and [master] sections accordingly.
[main] dns_alt_names=puppetmaster,puppet environmentpath = $confdir/environments [master] autosign=true
Note:
autosign=true
is used to automatically sign puppet agent certificates when puppet agent requests a catalog run from the puppet master for the first time.dns_alt_names
are mentioned so that when creating ssl certificates for the master itself, the names will be embedded to the certificate itself which is easy for the agent to find out that agent is connecting to the intended puppet master.environmentpath
will specify the puppet environments location. Default environment for puppet is/etc/puppet/environments/production
. In configuring puppet environments here we have used directory environments which are easier to use and will eventually replace config file environments completely from Puppet 4.0. onwards. - Before restarting the puppet master, clean all certificates, including puppet master’s certificate which is having its old DNS alt names.
> puppet cert clean --all
- Restart the puppet master for new configuration changes to take effect and to regenerate the certificate with the new
dns_alt_names
.
> service puppetmaster restart
Note: You can use the following command to check the puppet master’s new certificate and other certificates if any.
> puppet cert list --all + "puppetmaster.openstacklocal" (SHA256) 6B:F2:F4:96:5E:C5:70:E6:BB:1F:91:36:B6:91:40:B9:8D:A8:0D:D3:B2:9E:EC:64:63:8B:67:66:5C:B0:00:9C (alt names: "DNS:puppet", "DNS:puppetmaster", "DNS:puppetmaster.openstacklocal")
- Download git repo files
- git clone
wso2/puppet-modules
1 to a temp folder.
- git clone
- Prepare puppet modules
- Copy all modules in
../puppet-modules/modules
to/etc/puppet/environments/production/modules/
. - Install java module from puppet forge2. This forge module is used as a dependency to install a given JDK version in a given puppet agent prior to unpacking and starting a WSO2 product.
> puppet module install 7terminals-java
- Install
stdlib
module from puppet forge3.
> puppet module install puppetlabs-stdlib
- Copy all modules in
- Prepare hieradata
- Copy
../puppet-modules/hiera.yaml
to/etc/puppet
- Edit
hiera.yaml
file yaml data directory location as follows.
:yaml: :datadir: "/etc/puppet/hieradata/%{::environment}"
- Copy
../puppet-modules/hieradata
folder to/etc/puppet/
. - Rename
/etc/puppet/hieradata/dev
to/etc/puppet/hieradata/production
.
- Copy
- Prepare site.pp manifest
- Copy
../puppet-modules/manifests/site.pp
to/etc/puppet/environments/production/manifests/
.
- Copy
- Prepare files/packs
Here we are copying the relevant product pack and JDK installation pack to puppet master’s relevant puppet modules.
- Copy the pack file (e.g.
wso2am-1.9.1.zip
) to../modules/wso2am/files/
. - Create folder
../modules/wso2base/files
. - Copy jdk installation file (e.g.
jdk-7u79-linux-x64.gz
) to../modules/wso2base/files
. - Add jdk version (home) and filename information to
/etc/puppet/hieradata/production/common.yaml
.
- Copy the pack file (e.g.
- Login to puppet agent as the super user.
- Modify
/etc/puppet/puppet.conf
file of puppet agent appending the following to the [main] section.
[main] server = puppet
Task 3 - Set facter variables and perform a puppet agent run
Following sample files in references section are for APIM 1.9.1 [4], [5].
- Modify the deployment.conf file according to your product.
e.g. file contentsproduct_name=wso2am product_version=1.9.1 product_profile=default vm_type=openstack
- Login to puppet agent as super user.
- Copy the modified file deployment.conf -gt;
/opt/deployment.conf
. - Copy the file
setup.sh
[5] -gt;/opt/setup.sh
and execute the following command.
> chmod 755 setup.sh
- Run following command to setup WSO2 product instance.
> ./setup.sh
An instance of a standard WSO2 product given version of your choice will be installed. E.g. CARBON_HOME
-gt; /mnt/10.0.2.89/wso2am-1.9.1
. For APIM 1.9.1 the complete deployment time (deployment + server startup) will be approximately 1 minute.
FAQ
Puppet master/agent communication errors
Refer to this blog post for common errors and relevant fixes.
How can I view the wso2carbon.log?
Each WSO2 product you have selected to install in puppet agent will be extracted to a directory structure as follows.
/mnt/<IP address>/<product_name>-&ly;product_version> e.g.: CARBON_HOME -gt; /mnt/10.0.2.89/wso2am-1.9.1
You can find the log file in the above CARBON_HOME/repository/logs/wso2carbon.log
location.
How can I change the JDK version?
You can change the JDK version for a given WSO2 product instance as follows. It is a matter of editing the common yaml data file for the given puppet environment by changing the jdk home and jdk file name.
- Login to puppet master as the super user.
- Go to
/etc/puppet/hieradata/production/
. - Edit the
common.yaml
file by including the jdk installation directory and jdk pack file name. - Add the relevant JDK pack file to (e.g.:
jdk-7u80-linux-x64.tar.gz
) to../modules/wso2base/files
. - Login to puppet agent as the super user.
- Run following command to set up WSO2 product instance with the new JDK version.
> ./setup.sh
E.g. java_install_dir: /mnt/jdk-7u80 java_source_file: jdk-7u80-linux-x64.tar.gz
How can I change JVM parameters?
You have to change the common yaml data file at wso2 directory.
- Login to puppet master as the super user.
- Go to
/etc/puppet/hieradata/production/wso2/
. - Edit the
common.yaml
file as follows according to your required values.
E.g.: # JVM settings wso2::jvm : xms: 256m xmx: 1024m max_perm_size: 256m
- Login to puppet agent as the super user.
- Run following command to set up WSO2 product instance with new JVM parameters.
> ./setup.sh
Conclusion
In this tutorial you have learned how to use WSO2 Puppet Modules in automating the deployment of your WSO2 product of interest in a given puppet agent instance. We have used puppet master/agent architecture throughout this tutorial to automate the deployment of a standalone WSO2 product.
This is the first step in understanding the capabilities of WSO2 Puppet Modules. Through these puppet modules you will be able to automate complex deployment setups in a short period of time.
References
- [1] https://github.com/wso2/puppet-modules
- [2] https://forge.puppetlabs.com/7terminals/java
- [3] https://forge.puppetlabs.com/puppetlabs/stdlib
- [4] deployment.conf
product_name=wso2am product_version=1.9.1 product_profile=default vm_type=openstack
- [5] setup.sh
#!/bin/bash echo "#####################################################" echo " Starting cleanup " echo "#####################################################" ps aux | grep -i wso2 | awk {'print $2'} | xargs kill -9 #rm -rf /mnt/* sed -i '/environment/d' /etc/puppet/puppet.conf echo "#####################################################" echo " Setting up environment " echo "#####################################################" rm -f /etc/facter/facts.d/deployment_pattern.txt mkdir -p /etc/facter/facts.d while read -r line; do declare $line; done < deployment.conf echo product_name=$product_name >> /etc/facter/facts.d/deployment_pattern.txt echo product_version=$product_version >> /etc/facter/facts.d/deployment_pattern.txt echo product_profile=$product_profile >> /etc/facter/facts.d/deployment_pattern.txt echo vm_type=$vm_type >> /etc/facter/facts.d/deployment_pattern.txt echo "#####################################################" echo " Installing " echo "#####################################################" puppet agent --enable puppet agent -vt puppet agent --disable
- [6] https://suhan-opensource.blogspot.com/2014/10/puppet-master-agent-communication-errors.html