WSO2Con Insights: Experian Uses WSO2 to Uncover Credit Intelligence

Some call Experian a credit score checking service, but that would perhaps be an injustice: this company, which now counts some 17,000 people among its employees, is the credit information company. So deeply ingrained are they that in certain countries, it’s common to be told “Go talk to Experian” when you have a problem with your credit. Nor does it stop there. Experian’s products have long since expanded beyond credit and into everything from financial education to digital user analytics: it’s now a business with revenues in the billions of dollars. Experian has a very interesting set of needs. Day in and day out, customers arrive at Experian looking not only for credit reports, but for financial advice. Experian, analyzing their spending patterns and the ripple effects of those, is in a position to tell customers what to buy, what cards to keep, how to handle their bank accounts and loans, and a myriad of other details. In his talk at WSO2Con EU 2015 Rafael Garcia-Navarro, Head of Analytics at Experian explained how shifting from huge volume/low speed batch data processing to small volume/high speed data execution, helped them get their big data into shape.

The problem of real-time

Given the nature of what they do, Experian needs a lot of intelligence and data analysis power. In the world of credit intelligence, everything is linked - from where a user votes to the loans they’ve taken to the smartphone plan that he or she is on. In the past, they would process vast amounts of data offline and use that to make analyses. To this, Experian added a requirement: real-time operation - defined by them as systems that could take data from marketing channels, process and react with the required information under the average human reaction time of 200 milliseconds. More specifically, they needed systems that detect patterns at very high speeds, passing data in such a way that as to enable the full machinery to deliver complete results in under 200 milliseconds. This is where the WSO2 Complex Event Processor comes into the picture. Experian were working with some serious names in data analytics - like Google - and they began using the WSO2 CEP to analyze the customer data in real-time. The first step, is taking log files from digital platforms at the user level - cookies, if you will - to develop batch prediction models which help them decide what to promote to different users. The next step was to move out of purely historical data. Experian developed a Java application that simulates Google data; this data streams into WSO2 CEP. “What happens there is Siddhi is running the queries to identify the events that are relevant for further analysis, and driving that in into a Java-based platform,” said Garcia-Navarro “We take the latest events that we’ve identified from the streaming application, and we take those events to re-run the score with the latest information that is available to users, and re-optimizing that with MarketSwitch.” The system would constantly re-examine their data, updating it and fine-tuning it with the latest information, and drive the final, optimised decision back for execution on the marketing platforms. The challenge? In order to keep the whole system’s operation under 200 milliseconds, this particular sub-system had to do all of this at a mere 50 milliseconds. That’s a staggeringly small amount of time. After a pause, he added, “This 50 milliseconds has now been brought down to between 3 and 5 milliseconds.”

From code to credit

WSO2’s involvement began, ironically, not in the field of marketing analytics, but with analyzing credit risk. Experian had a product (now called PowerCurve) traditionally built for mainframes in the credit risk space; it allowed credit risk analysts to design business rules visually. They wanted to use this along with MarketSwitch to examine a user’s propensity to buy something. After the initial QuickStart program, Experian’s internal integrator - they have a team set aside for this - took it to the rest of the company. Even within Experian’s ocean of established technology stacks and software, the WSO2 CEP made a splash big enough to be a critical product. The first implementation connected to WSO2 CEP through WSO2 ESB. Later iterations directly connected to the Siddi processing engine. Experian likes the way WSO2 has worked for them on this.

“We explored all the typical suspects,” said Garcia-Navarro. “The CEP world is well known, and CEP for high-frequency trading had been in use for years. We explored all those commercial providers, but we chose WSO2 for three key reasons:

The first is because it’s open source. We believe that whenever possible we need to start embracing open source much more widely in business.

The second one is the depth of knowledge of the support provided. WSO2 takes a lot of pride in their support model; they claim - rightly - that they don’t have pre-engineers, but engineers who work on on the product providing the support needed for clients. And when you start working with them you see the depth of skills and expertise that they have. That’s a big plus for us.

The final one is the depth of offerings. CEP we’ve built the prototype for and implemented in house in our data centers and infrastructure. We’re starting to look into many aspects - the next one we’re looking into is ESB, but not the only one.“

Right now, Experian is pushing Complex Event Processor to the limits. Because of the nature of their business, they’re heavily interested in the next steps that we take with CEP and some of the new things we’re working on in the data and analytics space. For more information on Experian’s work with WSO2, view Rafael’s presentation at WSO2Con EU 2015.