Tag Archives: WSO2 Data Services Server

WSO2Con Insights – AlmavivA Adopts Lean Approach to Public Administration with WSO2

The Italian Ministry of Economy was looking for a complete transformation in data management by redefining and organizing its own data, so that information of millions of employees of the Italian Public Administration would be unique and certified.

The proposed system spelt the integration of two main IT systems in the Ministry; one that handles personal data, and a second that handles economic data, so that the system would have one single point of management, and serve applications regarding salaries and personal data as a self-service for the Italian public sector employees.

The Ministry approached AlmavivA Group, Italy’s number one Information and Communication Technology provider, for a solution. Guiseppe Bertone, Solution Architect at AlmavivA S.p.A. said during his session at WSO2Con 2014 EU, in Barcelona, Spain that AlmavivA designed and proposed an ad hoc master data management (MDM) solution for the Ministry, based on WSO2 products to manage the data of 2.6 million employees.

Picking the Best Product Solution

He said that there was a set criteria that AlmavivA and their client listed out prior to choosing the right products and platform for the project. Some of the critical features were interoperability with existing IT components, high modularity, optimized for performance, and most importantly, open source. Comparing pre-built product solutions available in the market, Bertone and his team made a decision to use WSO2 products for the entire solution.

“WSO2 products fit the requirement. You can enable only the components that you need, and leave the rest of it out, unlike in pre-built solutions,” he said.

He added that there were many redundant repositories within the Ministry IT systems; datasets needed to be optimized and integrated with external systems, and a migration workflow for the existing data had to be defined.

The reference architecture for the MDM solution included interface, events, security, and data quality components, as well as the repository layer, which consists of four databases; master data, metadata, historical data and reference data.

The AlmavivA project ‘Anagrafca Unica’, roughly translating to ‘Unique Repository’, was initiated in March 2012.

The WSO2 Advantage

The mapped reference architecture was a total solution platform based on a set of WSO2 products;

WSO2 Enterprise Service Bus (ESB) for interface services, the WSO2 Data Services Server (DSS) to access the repository layer and manage all life cycle services, WSO2 Identity Server (IS) as the security and identity component, WSO2 Message Broker (MB) for communication between applications, WSO2 Governance Registry (G-REG) to store configurations of all components, and the WSO2 Business Activity Monitor (BAM) to monitor services across the entire MDM solution. OracleDB is used as the repository layer.

With BAM being easily integrated to other WSO2 products, AlmavivA simply had to install only a specific BAM load inside each component, so that the statistics and real-time performance could be monitored. An additional console was added as an UI for the system’s custom procedures.

Another advantage of using WSO2 products was brought to light during the development stage; “Many aspects of WSO2 products can be simply configured from the web UI, or the developer studio for all WSO2 components. It’s really useful and easy to use,” explained Bertone.

In a covalent situation such as this, WSO2 deploys Carbon Apps. By creating a carbon app, a single file consisting of all components is created, so that once the file is deployed, the server knows which components to take, according to Bertone. “This is useful because once you have a system like this you can integrate it with an application cycle management solution already present in the customer environment, like we did,” he says. “We have now created a console where with a single click, the customer can pass from staging to production.”

AlmavivA is looking to expand Anagrafica Unica across the country to include all employees of the Italian Public Administration sector in the system, bringing the total user count to 3.5 million. Bertone and his team are also looking to serve data to external systems, such as the Ministry of Health, with more government institutions being added along the way.

For more information on AlmavivA’s development of the Master Data Management System, view the recording of Bertone’s WSO2Con EU presentation.

Pull/Push Data From Central Datacenter to Reduce Deployment Complexity

The Pattern we describe in this post will be useful to organizations that operate with a central master datacenter together with distributed applications in geographically diverse locations. We can take as examples the retail sector where an organization runs a chain of many stores, hospitality sector with many hotels and restaurants, or the healthcare sector with many hospitals and pharmacies.

Synchronizing data between the distributed agent repositories and the central master repository is a common requirement for businesses of this nature. Often the two-way synchronization must occur on at least a daily basis to keep all the systems up-to-date. Transaction data has to come from the agent data stores to the master store; master data and reconciled data has to go back to the agent data stores from the master data store.

A common approach to implement the above scenario is to run a periodic process within each agent data store, which connects with a process running in the central data store and creates a channel to transfer between data stores. This approach requires a large-scale system change, requiring systematic changes to both the central system and each agent store system. It may require coordination between agent stores to avoid overwhelming the central store.  Each store may even have to purchase new hardware and the associated costs can quickly scale upward depending on the number of connected stores. IT staff may be needed to look after each and every store to keep the system running smoothly. Costs and expertise requirements mount quickly with this architecture.

How can we reduce the deployment and management complexity and keep costs reasonable?  WSO2 has identified a solution pattern that pulls and pushes data from the main datacenter, without installing additional components and initiating periodic processes in the agent data stores.

Diagram 1

This pattern consists of a central process that schedules and connects to each agent data stores using a data connectivity technology (e.g. JDBC, ADO) and directly synchronizes the data (e.g. RDBMS) running in the store. The WSO2 middleware platform, specifically the WSO2 ESB and the WSO2 Data Services Server, provides OOTB functionality to implement the above pattern.  These products are deployed a the central datacenter location.

Diagram 2

WSO2 ESB scheduled tasks are configured to kick off the synchronization task, based on the frequency of the synchronization needed. These timer tasks invoke data services deployed in the WSO2 Data Services Server, which provide a CRUD (Create/Read/Update/Delete) service interface directly against the agent data repository. The WSO2 Data Services Server is capable of executing the SQL queries or calling a stored procedure of the agent data store as required to implement the required CRUD operations. WSO2 ESB will update the sub-system data store with data coming through the data services and will push the data back through to the master store through the same data services by reading from sub-systems. Built-in mediation features of the WSO2 ESB such as transformation and routing can be used to convert messages between different data models as well as route to relevant sub-systems, kick off additional events or processes, and so forth.

This pattern is suitable for both NRT (Near Real Time) and batch synchronization requirements, whichever is best suited for the organization that runs these type of distributed deployments.

Asanka Abeysinghe, Director of Solutions Architecture
Asanka’s blog: http://asanka.abeysinghe.org/