How to perform Data Mapping via the Smooks Editor in WSO2 Carbon Studio
- Malith Dhanushka
- Software Engineer - WSO2
Prerequisites
Objective
The purpose of this tutorial is to demonstrate how to perform data mapping via the Smooks Editor in WSO2 Carbon Studio.
Applies to
Carbon Studio | 1.0.11 |
Eclipse | Helios |
JDK | 1.6 |
OS | Ubuntu 10.10 |
Content
- Introduction
- How to Create a Smooks Configuration Artifact
- Sample Data Mapping Scenario
- How to Map CSV-XML
1 Introduction
1.1 What is Smooks? (As described in Smooks home page)
Smooks is an extensible Java framework for building applications for processing XML and non XML data (CSV, EDI, Java etc) .
Smooks can be used to:
1. Transformation
Perform a wide range of Data Transforms. Supports many different Source and Result types -XML/CSV/EDI/Java/JSON to XML/CSV/EDI/Java/JSON.
2. Java Binding
Bind into a Java Object Model from any data source (CSV, EDI, XML, Java, JSON etc).
3. Huge Message Processing
Process huge messages (GBs) - Split, Transform and Route message fragments to destinations such as JMS, File, Database etc. Route multiple message formats to multiple destinations in a single pass over a message.
4. Message Enrichment
Enrich a message with data from a Database, or other Datasources.
5. ORM Based Message Persistence
Use an entity persistence framework (like Ibatis, Hibernate or any JPA compatible framework) to access a database and use it's query language or CRUD methods to read from it or write to it.
6. Combine
Perform Extract Transform Load (ETL) operations by leveraging Smooks' Transformation, Routing and Persistence functionality.
1.2 Smooks tools in Carbon Studio
Smooks tools is a set of graphical tools for editing Smooks configuration file based on Eclipse developed by JBoss. We have integrated it in to WSO2 Carbon Studio as a third party tool.
1.3 Download and Install Carbon Studio
The Smooks tools has been included in to the WSO2 Carbon Studio since 1.0.5 version. You can download the WSO2 Carbon Studio from WSO2 download site.For a complete installation guide, please refer to Carbon Studio Installation Guide.
Please refer Smooks home page for more information on Smooks framework and it's features.
2. How to Create a Smooks Configuration Artifact
Follow the instructions given in the Product documentation link below to get some practice to create a Smooks configuration artifact.
3. Sample Data Mapping Scenario
3.1 How to map XML to XML
To do XML to XML mapping two models are required. One is to generate input model and the other to create a mapping model. For this example following are the input & output models.
model-input.xml
<order id='444'> <header> <customer number="555">Amila</customer> </header> <order-items> <order-item id='1'> <product>1</product> <quantity>2</quantity> <price>400</price> </order-item> </order-items> </order>
model-output.xml(Mapping Model)
<salesorder> <details> <orderid></orderid> <customer> <id></id> <name></name> </customer> </details> <itemList> <item> <id></id> <productId></productId> <quantity></quantity> <price></price> </item> </itemList> </salesorder>
Step 1
Create a CApp and save the input & output models in your file system.
Step 2
Create a Smooks configuration artifact as described in [2]. Make sure that you have entered the correct directory structure (/_system/config/custom) under "Registry Path To Deploy" unless at the runtime in ESB it will fail to find the registry resource.
Step 3
The following screen will now be shown.
Figure 01
Then set Input type as xml(Figure 02).
Figure 02
Step 4
Add the input model with data(Figure 03,Figure 04).
Figure 03
Figure 04
That will automatically create a input model in input model view(Figure 5).
Figure 05
Step 5
Then click on the add button on Input Task and then click on apply template(Figure 06).
Figure 06
set Message Type as XML and click on Next
Figure 07
Set “Create Template From:” as Sample XML & point the mapping mode then click on finish.
Figure 08
This will dynamically generate the graphical model as shown in figure 09.
Figure 09
Step 6
Then we can do the mapping. You can do the mapping by just connecting relevant elements or attributes by clicking on one element on the input model and connecting it to the corresponding element in the output model as shown below. Ensure to do the mapping from top to bottom or else the configuration will fail at run time. Figure 10 shows the complete mapping. So do the mapping exactly as shown in Figure 10 and save it.
Figure 10
Here is the resulting Smooks configuration xml.
Figure 11
Step 7
Next we can run the Smooks configuration xml and check weather the mapping is performed correctly. To do so, right click on Smooks configuration xml (smooks-config-xmltoxml) & select Run As > Smooks Run Configuration as shown in Figure 12.At the same time make sure you added the Smooks jars as mentioned earlier.
Figure 12
If your mapping was successful it will create the output as shown in figure 13.
Figure 13
3.2 ESB Smooks Mediation Support
WSO2 ESB supports Smooks mediation Since its 3.0.1 version.
Step 8
You can add this newly created Smooks configuration artifact in to a Proxy Service via ESB Tools. To do so,right click on CApp and click on New > Proxy Service as shown in Figure 14 and follow Figure 15.
Figure 14
Figure 15
Step 9
Then expand the proxy service as shown in Figure 16 and you will see there is a log mediator available,set its Log level “Full” from its properties.(This will show you you the request on ESB Shell).
Figure 16
Step 10
Then add a Smooks mediator by right clicking on Anonymous as shown in figure 17.
Figure 17
Step 11
By default the input parameter and output parameter will be set to xml so no need to change it. Then set the Configuration Key parameter from its properties by pointing the Smooks configuration xml you have just created as shown in Figure 18
Figure 18
Step 12
Then put a log mediator after Smooks mediator and set its log level as full as shown in figure 19.This will show the transformed request by Smooks in the shell.
Figure 19
Step 13
Then put a drop mediator to drop the message as shown in Figures 20 and Figure 21.
Figure 20
Figure 21
Step 14
Then export it as CAR by following the procedure shown in Figures 22 and 23.
Figure 22
Figure 23
Step 15
Next we are going to deploy our CAR file in to WSO2 ESB. You can Download WSO2 ESB from WSO2 download site .Run the WSO2 ESB and mange to open it on browser.
Step 16
Then add Governance Registry Server Role by click on Server Role under Configure tab as shown in Figure 24 since we have an artifact with server role set to Governance Registry.
Figure 24
Step 17
Then deploy the CAR file by clicking on add button in Application under Main tab as shown in Figure 25.
Figure 25
If the deployment has happened successfully you can see our proxy service is available in Web services list (Figure 26) & Smooks configuration xml is available under Registry (Figure 27).
Figure 26
Figure 27
Step 18
Then we are going to try out our Smooks transformation. In order to do that click on try out link on our proxy service as shown in Figure 28.
Figure 28
Put the following request in to the Request editor which has been redirected by previous link as shown in Figure 29.
request.xml
<order id='888'> <header> <customer number="111">Kasun</customer> </header> <order-items> <order-item id='1'> <product>1</product> <quantity>2</quantity> <price>900</price> </order-item> </order-items> </order>
Figure 29
If you switched on to Terminal which running ESB you can see the request has been transformed properly as shown in Figure 30.
Figure 30
4. How to map CSV to XML
To do CSV to XML mapping two things are required. One is to generate input model and the other to create a mapping model. And you want some data as input.
For this example following are the CSV fields, output model & input data. And in this scenario we are going to generate the xml template from XSD.
Fields
firstname,lastname,gender,age,country
Input Data(model-input.csv)
Chathuri,Wimalasena,Female,25,SriLanka Saminda,Wijerathne,Male,26,SriLanka Dakshitha,Rathnayaka,Female,24,SriLanka Harsha,Martin,Male,24,SriLanka
Mapping model(model-output.xsd)
<?xml version="1.0" encoding="UTF-8"?> <xs:schema xmlns:xs="https://www.w3.org/2001/XMLSchema"> <xs:element name="people" type="newpeople"/> <xs:complexType name="newpeople"> <xs:sequence> <xs:element name="person" type="newperson" maxOccurs="unbounded"/> </xs:sequence> </xs:complexType> <xs:complexType name="newperson"> <xs:sequence> <xs:element name="lastname" type="xs:string"/> <xs:element name="firstname" type="xs:string"/> <xs:element name="gender" type="xs:string"/> <xs:element name="age" type="xs:integer"/> <xs:element name="country" type="xs:string"/> </xs:sequence> </xs:complexType> </xs:schema>
Step 1
Create a CApp and save the input & output models in your file system(as .csv & .xsd).
Step 2
Create a Smooks configuration artifact as described in [2].
Step 3
Set Input Type as CSV (Figure 31).
Figure 31
Step 4
Add the fields given earlier in to Fields property under Input Configuration (Figure 32).
Set Input Type as CSV (Figure 31).
Figure 32
Step 5
Add the sample input data file by pointing .csv (model-input.csv) data file created earlier. This will dynamically creates the input model view as shown in Figure 33.
Figure 33
Step 6
Then add the mapping model xsd file (model-output.xsd) as shown Figure 34.
Figure 34
This will dynamically create a graphical model as shown in figure 35.
Figure 35
Step 7
Next do the mapping exactly as shown in Figure 36.At the same time make sure to proceed from top to bottom.
Figure 36
Smooks configuration relevant to this is shown in Figure 37.You can edit the Smooks configuration for further processing.
Figure 37
Step 8
After running the Smooks configuration it will create an output as shown in Figure 38.
Figure 38
Summery
In this tutorial we mainly discussed about Smooks Editor in WSO2 Carbon Studio, how to do a XML to XML mapping, Smooks mediator & how to transform a request by using Smooks processor in ESB. And then did a CSV to XML mapping. There are few other mapping methods available in Smooks Editor such as JSON,EDI,Java & etc which you can tryout .
References
Author
Malith Dhanushka, Intern, WSO2 Inc