2011/07/11
11 Jul, 2011

How to perform Data Mapping via the Smooks Editor in WSO2 Carbon Studio

  • Malith Dhanushka
  • Software Engineer - WSO2

Prerequisites

WSO2 Carbon Studio
WSO2 ESB

Objective

The purpose of this tutorial is to demonstrate how to perform data mapping via the Smooks Editor in WSO2 Carbon Studio.

Applies to

Carbon Studio 1.0.11
Eclipse Helios
JDK 1.6
OS Ubuntu 10.10

Content

1 Introduction

1.1 What is Smooks? (As described in Smooks home page)

Smooks is an extensible Java framework for building applications for processing XML and non XML data (CSV, EDI, Java etc) .

Smooks can be used to:

1. Transformation

Perform a wide range of Data Transforms. Supports many different Source and Result types -XML/CSV/EDI/Java/JSON to XML/CSV/EDI/Java/JSON.

2. Java Binding

Bind into a Java Object Model from any data source (CSV, EDI, XML, Java, JSON etc).

3. Huge Message Processing

Process huge messages (GBs) - Split, Transform and Route message fragments to destinations such as JMS, File, Database etc. Route multiple message formats to multiple destinations in a single pass over a message.

4. Message Enrichment

Enrich a message with data from a Database, or other Datasources.

5. ORM Based Message Persistence

Use an entity persistence framework (like Ibatis, Hibernate or any JPA compatible framework) to access a database and use it's query language or CRUD methods to read from it or write to it.

6. Combine

Perform Extract Transform Load (ETL) operations by leveraging Smooks' Transformation, Routing and Persistence functionality.

introduction

1.2 Smooks tools in Carbon Studio

Smooks tools is a set of graphical tools for editing Smooks configuration file based on Eclipse developed by JBoss. We have integrated it in to WSO2 Carbon Studio as a third party tool.

1.3 Download and Install Carbon Studio

The Smooks tools has been included in to the WSO2 Carbon Studio since 1.0.5 version. You can download the WSO2 Carbon Studio from WSO2 download site.For a complete installation guide, please refer to Carbon Studio Installation Guide.

Please refer Smooks home page for more information on Smooks framework and it's features.

2. How to Create a Smooks Configuration Artifact

Follow the instructions given in the Product documentation link below to get some practice to create a Smooks configuration artifact.

https://docs.wso2.org/display/DVS320/Creating+ESB+Artifacts#CreatingESBArtifacts-CreatingaSmooksConfigurationArtifact

3. Sample Data Mapping Scenario

3.1 How to map XML to XML

To do XML to XML mapping two models are required. One is to generate input model and the other to create a mapping model. For this example following are the input & output models.

model-input.xml

<order id='444'>
    <header>
        <customer number="555">Amila</customer>
    </header>
    <order-items>
        <order-item id='1'>
            <product>1</product>
            <quantity>2</quantity>
            <price>400</price>
        </order-item>
    </order-items>
</order>

model-output.xml(Mapping Model)

<salesorder>
    <details>
        <orderid></orderid>
        <customer>
            <id></id>
            <name></name>
        </customer>
    </details>
    <itemList>
        <item>
            <id></id>
            <productId></productId>
            <quantity></quantity>
            <price></price>
        </item>
    </itemList>
</salesorder>

Step 1

Create a CApp and save the input & output models in your file system.

Step 2

Create a Smooks configuration artifact as described in [2]. Make sure that you have entered the correct directory structure (/_system/config/custom) under "Registry Path To Deploy" unless at the runtime in ESB it will fail to find the registry resource.

Step 3

The following screen will now be shown.

Figure 01

Then set Input type as xml(Figure 02).

Figure 02

Step 4

Add the input model with data(Figure 03,Figure 04).

Figure 03

Figure 04

That will automatically create a input model in input model view(Figure 5).

Figure 05

Step 5

Then click on the add button on Input Task and then click on apply template(Figure 06).

Figure 06

set Message Type as XML and click on Next

Figure 07

Set “Create Template From:” as Sample XML & point the mapping mode then click on finish.

Figure 08

This will dynamically generate the graphical model as shown in figure 09.

Figure 09

Step 6

Then we can do the mapping. You can do the mapping by just connecting relevant elements or attributes by clicking on one element on the input model and connecting it to the corresponding element in the output model as shown below. Ensure to do the mapping from top to bottom or else the configuration will fail at run time. Figure 10 shows the complete mapping. So do the mapping exactly as shown in Figure 10 and save it.

Figure 10

Here is the resulting Smooks configuration xml.

Figure 11

Step 7

Next we can run the Smooks configuration xml and check weather the mapping is performed correctly. To do so, right click on Smooks configuration xml (smooks-config-xmltoxml) & select Run As > Smooks Run Configuration as shown in Figure 12.At the same time make sure you added the Smooks jars as mentioned earlier.

Figure 12

If your mapping was successful it will create the output as shown in figure 13.

Figure 13

3.2 ESB Smooks Mediation Support

WSO2 ESB supports Smooks mediation Since its 3.0.1 version.

Step 8

You can add this newly created Smooks configuration artifact in to a Proxy Service via ESB Tools. To do so,right click on CApp and click on New > Proxy Service as shown in Figure 14 and follow Figure 15.

Figure 14

Figure 15

Step 9

Then expand the proxy service as shown in Figure 16 and you will see there is a log mediator available,set its Log level “Full” from its properties.(This will show you you the request on ESB Shell).

Figure 16

Step 10

Then add a Smooks mediator by right clicking on Anonymous as shown in figure 17.

Figure 17

Step 11

By default the input parameter and output parameter will be set to xml so no need to change it. Then set the Configuration Key parameter from its properties by pointing the Smooks configuration xml you have just created as shown in Figure 18

Figure 18

Step 12

Then put a log mediator after Smooks mediator and set its log level as full as shown in figure 19.This will show the transformed request by Smooks in the shell.

Figure 19

Step 13

Then put a drop mediator to drop the message as shown in Figures 20 and Figure 21.

Figure 20

Figure 21

Step 14

Then export it as CAR by following the procedure shown in Figures 22 and 23.

Figure 22

Figure 23

Step 15

Next we are going to deploy our CAR file in to WSO2 ESB. You can Download WSO2 ESB from WSO2 download site .Run the WSO2 ESB and mange to open it on browser.

Step 16

Then add Governance Registry Server Role by click on Server Role under Configure tab as shown in Figure 24 since we have an artifact with server role set to Governance Registry.

Figure 24

Step 17

Then deploy the CAR file by clicking on add button in Application under Main tab as shown in Figure 25.

Figure 25

If the deployment has happened successfully you can see our proxy service is available in Web services list (Figure 26) & Smooks configuration xml is available under Registry (Figure 27).

Figure 26

Figure 27

Step 18

Then we are going to try out our Smooks transformation. In order to do that click on try out link on our proxy service as shown in Figure 28.

Figure 28

Put the following request in to the Request editor which has been redirected by previous link as shown in Figure 29.

request.xml

 <order id='888'>
    <header>
        <customer number="111">Kasun</customer>
    </header>
    <order-items>
        <order-item id='1'>
            <product>1</product>
            <quantity>2</quantity>
            <price>900</price>
        </order-item> 
    </order-items>
</order>

Figure 29

If you switched on to Terminal which running ESB you can see the request has been transformed properly as shown in Figure 30.

Figure 30

4. How to map CSV to XML

To do CSV to XML mapping two things are required. One is to generate input model and the other to create a mapping model. And you want some data as input.

For this example following are the CSV fields, output model & input data. And in this scenario we are going to generate the xml template from XSD.

Fields

firstname,lastname,gender,age,country

Input Data(model-input.csv)

Chathuri,Wimalasena,Female,25,SriLanka
Saminda,Wijerathne,Male,26,SriLanka
Dakshitha,Rathnayaka,Female,24,SriLanka
Harsha,Martin,Male,24,SriLanka

Mapping model(model-output.xsd)

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="https://www.w3.org/2001/XMLSchema">
	<xs:element name="people" type="newpeople"/>
	<xs:complexType name="newpeople">
		<xs:sequence>
			<xs:element name="person" type="newperson" maxOccurs="unbounded"/> 
		</xs:sequence>        
	</xs:complexType>	
        <xs:complexType name="newperson">
		<xs:sequence> 
			<xs:element name="lastname" type="xs:string"/> 
			<xs:element name="firstname" type="xs:string"/> 
			<xs:element name="gender" type="xs:string"/> 
			<xs:element name="age" type="xs:integer"/> 
			<xs:element name="country" type="xs:string"/>  
               </xs:sequence>        
	</xs:complexType> 
</xs:schema>

Step 1

Create a CApp and save the input & output models in your file system(as .csv & .xsd).

Step 2

Create a Smooks configuration artifact as described in [2].

Step 3

Set Input Type as CSV (Figure 31).

Figure 31

Step 4

Add the fields given earlier in to Fields property under Input Configuration (Figure 32).

Set Input Type as CSV (Figure 31).

Figure 32

Step 5

Add the sample input data file by pointing .csv (model-input.csv) data file created earlier. This will dynamically creates the input model view as shown in Figure 33.

Figure 33

Step 6

Then add the mapping model xsd file (model-output.xsd) as shown Figure 34.

Figure 34

This will dynamically create a graphical model as shown in figure 35.

Figure 35

Step 7

Next do the mapping exactly as shown in Figure 36.At the same time make sure to proceed from top to bottom.

Figure 36

Smooks configuration relevant to this is shown in Figure 37.You can edit the Smooks configuration for further processing.

Figure 37

Step 8

After running the Smooks configuration it will create an output as shown in Figure 38.

Figure 38

Summery

In this tutorial we mainly discussed about Smooks Editor in WSO2 Carbon Studio, how to do a XML to XML mapping, Smooks mediator & how to transform a request by using Smooks processor in ESB. And then did a CSV to XML mapping. There are few other mapping methods available in Smooks Editor such as JSON,EDI,Java & etc which you can tryout .

References

Author

Malith Dhanushka, Intern, WSO2 Inc

 

About Author

  • Malith Dhanushka
  • Software Engineer
  • WSO2