[WSO2Con EU 2017] Building Next Generation Banking Middleware at ING: The Role of a Lightweight ESB

  • By WSO2 Team
  • 7 Nov, 2017

Building Next Generation Banking Middleware at ING: The Role of a Lightweight ESB

play button

Building Next Generation Banking Middleware at ING: The Role of a Lightweight ESB

Miguel Garcia Enterprise Architect, ING Model Bank
Download Slides

Building banking technical stacks is not an easy task, but by definition there is a need for information integrity, performance, security, stability, availability and flexibility in order to offer the best customer experience. In a classical approach, vertical monolith stacks based on pure transactional SQL BBDDs worked well, but today banks are offering new capabilities, both to customers and partners, and need to compete globally. This scenario creates the need to design non-monolith, distributed systems that use collaboration and composition instead of orchestration; this needs to offer the same capabilities than classical ones and be open to a new customer - developers, who are transforming the technical landscape. The introduction of streams, API management, microservices, distributed computing, real-time capabilities, new deployment approaches, and the transition from old-school IT requires some well-defined architecture systems and components. This talk will focus on how WSO2 ESB plays a key role in this transformation thanks to its flexibility, performance, openness, stability, and low TCO.

Transcript

Hi, good afternoon. I’m trying to explain the evolution of the approach of middleware system. Just in a briefly, from the classic view of a bank to the new one. And which pattern we need to observe in order to maintain that middleware. In order, with the evolution of technology. 00:00 ▶

This is the classic picture of a bank, of course. All the banks have a super big host legacy and so on. It could be true. 0:37 ▶

But the reality is that we don’t have one big host. In practice we have a lot of big systems that are interconnected or were interconnected as silos. If you want to just analyze the functionality and why banks and other organizations, are not so complex, just visit bian.org to see for sample functionality landscape. The complexity is very high and the processes we have to support are very high as well. And that’s the reason, of course we need to, that’s the reason we have a lot of big hosts and we need to abort those banking systems to be able to adapt to our customers. The banks are offering services, we have PSD2 as our colleagues mentioned this morning, we have a lot of challenges we need to adapt to them. 0:50 ▶

The most recent approximation to the evolution of banking middleware are microsystems. It’s okay, but microsystem or micro-services are offering only an approach that is simple only in greenfield approaches. When you try to abort your system and put micro-services in place, you will have a lot of problems. For example, integrity of information, the processes needs a lot of traceability properties that need to be observed. And if our greenfield approach could work, but besides that you have a lot of legacy systems that need to work and you need to preserve that logistic system to have a lot of investment on it, okay. 1:55 ▶

So the first thing you need to follow, for us is to not oversimplify your problem. I have seen a lot of articles about microservices, the evolution of middleware that simplify a problem that really will arise when you try to put a lot of complexity processes on it. Other thing that is not real, is the fixed target architecture. You won’t see that in any situation. Most probably when you reach that super target architecture, you need to abort, or you have a change in you revelation and so on. That’s the reason for a sample the modulo in school talk about business modality, because the modality is a need to abort your architecture and your business. So the real goal is to be prepared for change, and design from change your architecture. Avoid big migration, a big bang is not good for customers and not good for your investment, and not good for your organization. Be prepared to abort your organization in micro-changes. That’s microservices could change, could help, but you need to organize this with some clear patterns. Don’t separate your customers from you. You must seek to be good for your customers. So your system needs to adapt to your needs. 3:01 ▶

Let’s put in place our old pattern that will help us a lot in that situation, is the BUS pattern. The bus pattern is not intended to put the broker in the center. It’s a pattern that allows composition by years of serving 3 common rules. Common communication infrastructure, common data model, your microservices need to talk to each other, okay, and you need to perform or offer a common structure so you can, every microservices can understand each other. And you need to put that in place, no matter if you have an ESB or not. 4:58 ▶

In fact, the BUS pattern is just to make that communication between building blocks possible. It’s just a way in which you can do composition. 5:52 ▶

Now, you can do composition in that way. In that case you could prepare a beautiful silos. 6:09 ▶

Or you can do composition in that way. The real composition is that one, so you will need a lot of the design to separate your building blocks in clear functional landscape. 6:19 ▶

When you try to join your building blocks, you can do it in several ways. Of course the first the sign, prepare all your functionality landscape clearly if not you are going to mix the functionality in several building blocks. You can trace you information in banks, it’s absolutely key. You can perform patterns to preserve the integrity of the information, and when you try to put this on technology, you have two options. First you design, your BUS pattern, but to implement this you have two options. A broker, or a broker-less approach. The goal is to provide a BUS pattern not ‘how do you do that’. This is a famous discussion between the ZeroMQ and RabbitMQ developers, discussing if you need a broker to perform our BUS or not. In that case our technical discussion between broker middleware. But remember when you use microservices for example, you are using REST APIs and events, and with those event streams. Events are so important to REST APIs and you need to normalize them as well. We are using something for example, the main events are business facts that happen, and you need to understand, or your building blocks need to understand those building blocks. In most cases when you need to implement, you will implement BUS patterns then with a broker, Rabbit, Kafka for example, but you could do in a broker less way as well. But closely if you talk about API most of the people think about the broker less approach, okay. 6:37 ▶

Broker-less approach requires a registry to locate the different applications. That’s one advantage and some drawbacks. Requires a lot of governance because if you don’t put, for example a single building block to access to any system, you will go into the famous problem that will do a lot of difficulty to maintain. For example, if you need to change your security library, you need to deploy all your microservices again. So don’t analyze all your options about being broker-less with a closed mind and obtain all the pros and cons. You can have as well the anti-pattern because you have more participants, but the simplicity is so good you will go to that option if you want composition. 8:34 ▶

The broker approach in that scenario, all your interactions happens around the broker. For example a Kafka or an ESB, and the building blocks in theory should be simple because you put all the trust and concern with the broker. And the changes are faster because you need to change for example your security, most probably you have some filters in the building blocks but the main filters are in the broker, okay. So in practice if you have legacy and you want to abort your architecture. 9:48 ▶

In practice what you have to do is to abort one hybrid approach. The hybrid approach, for example needs to be in that way in which the broker is a first order participant in the microservices approach. It should talk with the service discovery as well as any other participant and route the information when you need to talk with an older system or with a system that needs to be broker for any reliable reason, or for example, cannot release purposes. It’s by far the simpler approach release to put a broker in between your request and your new blocks, okay. 10:30 ▶

We are using WSO2 as a broker because we really like the simplicity, we have a lot of tests a lot of problems and because it would be so low that we use as any other building blocks. In fact we are not deploying is as a cluster, with a low balancing schema. And our bus is around all the companies, not only around the broker. 11:17 ▶

We are using as well to provide API gateways to outside the wall, and all the APIs are defined in our life cycle development and are deployed in our different building blocks to change our broker-less approach in terms API. It’s just by configuration, it’s very simple, okay. So we have the best approach in a broker-less scenario. Or if we need a broker authentic approach we have to change the configuration. 11:53 ▶

Some patterns that you need to put in place. We don’t recommend a too fast commit transactions, please avoid that if you don’t want to block all of your systems. Go to the Sagas patterns instead. It’s very good, it works very well in most situations, especially banking because most core banking system don’t observe protocols and you just need to put this in any block that is going to perform orchestration or co-ordination. Okay, it’s very important to develop a clear way to do so. Secure risks, its key to maintain the integrity of the information and the views to the customers especially in the retail scenario. For example, when you go to find more information in the core banking system, you need to perform a command you do in a clear integrity way and when you need to get information to the customer, you will use the views that you are creating mostly using events that are produced in your core databases. Streaming, try to move your old transactions that really could be moved to the streaming schema, try to move these transactions in a clear streaming way the main events I mentioned are key to maintain the decomposability of the building blocks and to create those streams. Okay. As we have a lot of classic system that is not a key to change, at the moment for example. We need to focus on things that matter related to our customers. With WSO2 ESB we have a lot of adaptation, we create events, we performed APIs to also adaptation and for us it’s a key component in the building block BUS schema. 12:32 ▶

It’s very flexible, a lot of requests in productions. This is a real case. The sensibility is key, the OSGI foundation is key for us. For example we develop a ready connector you have to, transitive dependency you need to manage as OSGI bundles, and its key that foundation really provides a modular way to provide connectors and APIs. Because APIs have to deploy by the team themselves. The registry is very useful as well to provide some flex in production. 14:59 ▶

And some lessons that we have learned, if you want to use that product, you need to have deep knowledge of the tricky things. For example what happens when your system goes down, what happens when the disks o the ESB are failing for example, how to deploy that, use the latest version as possible , maintain your own class mediators under control. Don’t put a lot, try to use, because the transitive dependencies are always there. But it’s true that they were very well, but don’t change the philosophy of the product to use only your class mediator. Use GMX, use clear templating for automatic API creation, don’t let the double offers to create APIs, it’s not necessary. Just create it automatically from Swagger, use access those two modules in the product to for example, check security on things that you need to maintain under control. And define your common message properties for stability, purity, security, according to your service. Even if you are using Rest, they are going to be put in heathers, your Rest APIs. 15:44 ▶

So to end, design for change, don’t over simplify your problems, analyze your options and be firm with your pros and cons, and introduce new patterns that you will need for sure in your architecture as I mentioned before. Because the flexibility that you are going to achieve with building blocks and microservices is very high, it’s good, but if you do without control you are going to introduce micro-silos and a complexity that is impossible to manage. 17:16 ▶

So thank you, and if you have questions. Okay, thank you very much. What? Sorry. Yep. What made, I don’t understand. To create events, you publish those events in Kafka, WSO2 can publish a Kafka if you have the right person. But we are not mixing, we have found a clear definition between the streams and we operate with another kind of broker. Mainly the WSO2 in our case is used in transactional or in scenarios in which you require a response, or in scenarios you need to create events to the Kafka parts. But for us there is no direct relationship between the bus to that kind of interaction and the event bus. In fact in the bus we have RabbitMQ, because Kafka was not ready for the production and we have at the same time \ those two as well. So for us there is no relation between one and another. Any other question? Okay, thank you very much. 17:58 ▶

Presenter

Miguel Garcia Enterprise Architect, ING Model Bank

Miguel is an Enterprise Architect at ING Model Bank. He’s worked in innovation and technology for more than 20 years, especially focused on middleware in large and mission critical platforms, designing and rolling them out. In his role at ING, he is involved in efforts to build their next generation banking middleware.