How to perform Data Mapping via the Smooks Editor in WSO2 Carbon Studio
By Malith Dhanushka
- 11 Jul, 2011
The purpose of this tutorial is to demonstrate how to perform data mapping via the Smooks Editor in WSO2 Carbon Studio.
- How to Create a Smooks Configuration Artifact
- Sample Data Mapping Scenario
- How to Map CSV-XML
1.1 What is Smooks? (As described in Smooks home page)
Smooks is an extensible Java framework for building applications for processing XML and non XML data (CSV, EDI, Java etc) .
Smooks can be used to:
Perform a wide range of Data Transforms. Supports many different Source and Result types -XML/CSV/EDI/Java/JSON to XML/CSV/EDI/Java/JSON.
2. Java Binding
Bind into a Java Object Model from any data source (CSV, EDI, XML, Java, JSON etc).
3. Huge Message Processing
Process huge messages (GBs) - Split, Transform and Route message fragments to destinations such as JMS, File, Database etc. Route multiple message formats to multiple destinations in a single pass over a message.
4. Message Enrichment
Enrich a message with data from a Database, or other Datasources.
5. ORM Based Message Persistence
Use an entity persistence framework (like Ibatis, Hibernate or any JPA compatible framework) to access a database and use it's query language or CRUD methods to read from it or write to it.
Perform Extract Transform Load (ETL) operations by leveraging Smooks' Transformation, Routing and Persistence functionality.
1.2 Smooks tools in Carbon Studio
Smooks tools is a set of graphical tools for editing Smooks configuration file based on Eclipse developed by JBoss. We have integrated it in to WSO2 Carbon Studio as a third party tool.
1.3 Download and Install Carbon Studio
The Smooks tools has been included in to the WSO2 Carbon Studio since 1.0.5 version. You can download the WSO2 Carbon Studio from WSO2 download site.For a complete installation guide, please refer to Carbon Studio Installation Guide.
Please refer Smooks home page for more information on Smooks framework and it's features.
2. How to Create a Smooks Configuration Artifact
Follow the instructions given in the Product documentation link below to get some practice to create a Smooks configuration artifact.
3. Sample Data Mapping Scenario
3.1 How to map XML to XML
To do XML to XML mapping two models are required. One is to generate input model and the other to create a mapping model. For this example following are the input & output models.
<order id='444'> <header> <customer number="555">Amila</customer> </header> <order-items> <order-item id='1'> <product>1</product> <quantity>2</quantity> <price>400</price> </order-item> </order-items> </order>
<salesorder> <details> <orderid></orderid> <customer> <id></id> <name></name> </customer> </details> <itemList> <item> <id></id> <productId></productId> <quantity></quantity> <price></price> </item> </itemList> </salesorder>
Create a CApp and save the input & output models in your file system.
Create a Smooks configuration artifact as described in . Make sure that you have entered the correct directory structure (/_system/config/custom) under "Registry Path To Deploy" unless at the runtime in ESB it will fail to find the registry resource.
The following screen will now be shown.
Then set Input type as xml(Figure 02).
Add the input model with data(Figure 03,Figure 04).
That will automatically create a input model in input model view(Figure 5).
Then click on the add button on Input Task and then click on apply template(Figure 06).
set Message Type as XML and click on Next
Set “Create Template From:” as Sample XML & point the mapping mode then click on finish.
This will dynamically generate the graphical model as shown in figure 09.
Then we can do the mapping. You can do the mapping by just connecting relevant elements or attributes by clicking on one element on the input model and connecting it to the corresponding element in the output model as shown below. Ensure to do the mapping from top to bottom or else the configuration will fail at run time. Figure 10 shows the complete mapping. So do the mapping exactly as shown in Figure 10 and save it.
Here is the resulting Smooks configuration xml.
Next we can run the Smooks configuration xml and check weather the mapping is performed correctly. To do so, right click on Smooks configuration xml (smooks-config-xmltoxml) & select Run As > Smooks Run Configuration as shown in Figure 12.At the same time make sure you added the Smooks jars as mentioned earlier.
If your mapping was successful it will create the output as shown in figure 13.
3.2 ESB Smooks Mediation Support
WSO2 ESB supports Smooks mediation Since its 3.0.1 version.
You can add this newly created Smooks configuration artifact in to a Proxy Service via ESB Tools. To do so,right click on CApp and click on New > Proxy Service as shown in Figure 14 and follow Figure 15.
Then expand the proxy service as shown in Figure 16 and you will see there is a log mediator available,set its Log level “Full” from its properties.(This will show you you the request on ESB Shell).
Then add a Smooks mediator by right clicking on Anonymous as shown in figure 17.
By default the input parameter and output parameter will be set to xml so no need to change it. Then set the Configuration Key parameter from its properties by pointing the Smooks configuration xml you have just created as shown in Figure 18
Then put a log mediator after Smooks mediator and set its log level as full as shown in figure 19.This will show the transformed request by Smooks in the shell.
Then put a drop mediator to drop the message as shown in Figures 20 and Figure 21.
Then export it as CAR by following the procedure shown in Figures 22 and 23.
Next we are going to deploy our CAR file in to WSO2 ESB. You can Download WSO2 ESB from WSO2 download site .Run the WSO2 ESB and mange to open it on browser.
Then add Governance Registry Server Role by click on Server Role under Configure tab as shown in Figure 24 since we have an artifact with server role set to Governance Registry.
Then deploy the CAR file by clicking on add button in Application under Main tab as shown in Figure 25.
If the deployment has happened successfully you can see our proxy service is available in Web services list (Figure 26) & Smooks configuration xml is available under Registry (Figure 27).
Then we are going to try out our Smooks transformation. In order to do that click on try out link on our proxy service as shown in Figure 28.
Put the following request in to the Request editor which has been redirected by previous link as shown in Figure 29.
<order id='888'> <header> <customer number="111">Kasun</customer> </header> <order-items> <order-item id='1'> <product>1</product> <quantity>2</quantity> <price>900</price> </order-item> </order-items> </order>
If you switched on to Terminal which running ESB you can see the request has been transformed properly as shown in Figure 30.
4. How to map CSV to XML
To do CSV to XML mapping two things are required. One is to generate input model and the other to create a mapping model. And you want some data as input.
For this example following are the CSV fields, output model & input data. And in this scenario we are going to generate the xml template from XSD.
Chathuri,Wimalasena,Female,25,SriLanka Saminda,Wijerathne,Male,26,SriLanka Dakshitha,Rathnayaka,Female,24,SriLanka Harsha,Martin,Male,24,SriLanka
<?xml version="1.0" encoding="UTF-8"?> <xs:schema xmlns:xs="https://www.w3.org/2001/XMLSchema"> <xs:element name="people" type="newpeople"/> <xs:complexType name="newpeople"> <xs:sequence> <xs:element name="person" type="newperson" maxOccurs="unbounded"/> </xs:sequence> </xs:complexType> <xs:complexType name="newperson"> <xs:sequence> <xs:element name="lastname" type="xs:string"/> <xs:element name="firstname" type="xs:string"/> <xs:element name="gender" type="xs:string"/> <xs:element name="age" type="xs:integer"/> <xs:element name="country" type="xs:string"/> </xs:sequence> </xs:complexType> </xs:schema>
Create a CApp and save the input & output models in your file system(as .csv & .xsd).
Create a Smooks configuration artifact as described in .
Set Input Type as CSV (Figure 31).
Add the fields given earlier in to Fields property under Input Configuration (Figure 32).
Set Input Type as CSV (Figure 31).
Add the sample input data file by pointing .csv (model-input.csv) data file created earlier. This will dynamically creates the input model view as shown in Figure 33.
Then add the mapping model xsd file (model-output.xsd) as shown Figure 34.
This will dynamically create a graphical model as shown in figure 35.
Next do the mapping exactly as shown in Figure 36.At the same time make sure to proceed from top to bottom.
Smooks configuration relevant to this is shown in Figure 37.You can edit the Smooks configuration for further processing.
After running the Smooks configuration it will create an output as shown in Figure 38.
In this tutorial we mainly discussed about Smooks Editor in WSO2 Carbon Studio, how to do a XML to XML mapping, Smooks mediator & how to transform a request by using Smooks processor in ESB. And then did a CSV to XML mapping. There are few other mapping methods available in Smooks Editor such as JSON,EDI,Java & etc which you can tryout .
Malith Dhanushka, Intern, WSO2 Inc