Today, successful enterprises rely heavily on underlying software applications. To meet prevalent
diverse business requirements, enterprises have to pick and choose different software applications
that have been created with disparate technologies and built by different vendors. When the
enterprise needs to fulfil a given set of business requirements, most often they have to make these
disparate software applications work together to produce a unified set of functionality. The task
of plumbing different software applications and forming new software solutions is referred to as
'enterprise integration'.
Enterprises often encounter many challenges when dealing with applications built by varied vendors;
one of the most difficult, yet common problems is that some of these applications are based on
proprietary standards and others built with open standards. These challenges can be addressed with
the use of architectural styles and enterprise integration technologies that lie within a comprehensive
integration platform.
2. Architectural Styles for Integration
There are several integration styles that can be used to integrate two or more software applications.
In the publication "Enterprise Integration Patterns" by Gregor Hohpe, these integration styles have
been illustrated under four main categories: File Transfer, Shared Databases, Remote Procedure
Invocations, and Messaging (Figure 1).
Managed File Transfer has been around ever since the early stages of enterprise integration where
one application writes a file that the other application reads and visa versa. The file format, standards,
location, privileges and read/write coordination must be negotiated between the two applications
beforehand [comma-separated values (CSV), electronic data interchange (EDI), electronic protected
health information (ePHI), etc. are still used in enterprise IT].
With the Shared Database approach, software applications can use a common database to share data among each other.
In Remote Procedure Invocation (remote procedure calls, or RPC), a given application can expose
some functionality via a well-defined interface and other applications can invoke such functionality via
a remote invocation. Technologies, such as common object request broker architecture (CORBA), were
popular as an RPC-style integration methodology prior to SOA and web service technologies.
With Message-based integration style, all applications connect to a unified messaging bus and
exchange data and invoke functionalities via messages. It allows complete decoupling of applications
and enables high speed, asynchronous/synchronous, program-to-program communication with
reliable delivery message-oriented middleware (MoM), service-oriented architecture based on web
services and an enterprise service bus.
While these integration styles are actively used in most integration scenarios, the enterprise
integration space has evolved around these styles.
3. Evolution of Technologies for Enterprise Integration
Over several decades, enterprise integration technologies have leveraged one or more of the
integration styles that were discussed earlier. The evolution of enterprise integration middleware was
driven by the evolution of enterprise requirements.
3.1 Homegrown Integration Solutions/Custom Development
In the early stages of enterprise integration, most organizations met their integration requirements
by implementing in-house integration middleware. With this approach, integration middleware was
developed in-house using ad hoc and proprietary protocols.
Over time, with the addition of multiple software systems, the home-grown middleware system
becomes increasingly complex and ultimately fails. Most often than not, these systems are monolithic,
not reusable, and built in an ad hoc manner with point-to-point connectivity between each application.
With this approach, organizations end up focusing more on solving its integration problems rather
than focusing on the business functionality of the solution.
3.2 Enterprise Application Integration (EAI)
It is near impossible to build and maintain a homegrown integration solution for complex integration
scenarios that involve several applications and systems. Therefore, there was a need for a dedicated
integration middleware framework that can centralize and facilitate all integration requirements of the
infrastructure.
To this end, an EAI solution helps to seamlessly integrate applications and achieve a given business
objective. Initially, EAI solutions were designed based on hub-spoke architecture and later with bus
architecture.
3.2.1 Hub/Spoke Architecture
EAI hub/spoke architecture simplifies the integration of various applications by avoiding point-to-point
interactions and allows the loose coupling of applications. Since the hub is the centralized component,
this reduces the overhead of maintaining multiple configuration repositories. As illustrated in Figure 2,
the central broker (hub) connects all applications using adapters (spoke) for each software system.
Figure 2: EAI Hub-Spoke Architecture
Yet, the hub-spoke architecture has inherent limitations; the centralized hub introduces a single point
of failure, and with the heavy load, the hub can be the main bottleneck of the entire infrastructure.
Another drawback of this architecture is that most of the solution was built with proprietary
technologies and focussed on a particular business domain. Hence, it was difficult to adopt EAI as a
common integration middleware.
3.2.2 Bus Architecture
On the other hand, ‘bus architecture’ as illustrated in Figure 3 looks to address some of the limitations
prevalent with EAI hub-spoke.
Figure 3: EAI Bus Architecture
Bus architecture solves the scalability issues of the hub-spoke architecture as the centralized
messaging bus can be scaled horizontally. However, the distributed integration tasks complicates
and increases maintenance and troubleshooting overhead. Again, the non-standard base and the
proprietary nature of the EAI bus architecture eventually hampered the adoption of EAI-bus.
3.2.3 SOA and the Advent of Enterprise Service Bus
From the 2000s, SOA has become increasingly popular as an architectural strategy and many
organizations adopted SOA as the foundation of their enterprise architecture. SOA was realized in the
form of web services and web services became a primary part of implementing software solutions.
This has revolutionized the way enterprises approach their software solutions today. The heavy
adaptation of SOA leads the industry to seek a solid infrastructure to build a robust SOA. In this
context, software applications are more or less replaced with web services and the necessary
infrastructure is required to do the plumbing among these services to form various business solutions.
As with software applications, the interactions between services can be implemented with pointto-point
architecture, which will have the same set of problems and limitations that were discussed
earlier. With SOA, the bus architecture of the EAI has further evolved to a new architectural concept,
the ‘enterprise service bus’ (ESB). The idea of using an ESB is to provide a unified, standard-based
messaging layer where all the interactions between services take place.
4. ESB - The Enterprise Integration Backbone
ESB is designed as the middleware layer that enables interoperability among heterogeneous systems
and services using the SOA model. ESBs are often fully compliant with open standards such as WS-*
standards and encourage the organization to implement business functionalities as services. It acts
as the main messaging backbone in any SOA implementation. Furthermore, unlike traditional EAI, the
messaging layer is lightweight and can be horizontally scaled as per the organization’s needs.
Figure 4: Point-to-Point Integration versus ESB: Using ESB to Eliminate P2P Integration Nightmare
As explained in Figure 4, an ESB eliminates point-to-point integration of the organization and
integrates all disparate systems with the bus architecture. Generic ESB functionalities can be listed as
follows:
- Message mediation - Manipulates the message content, direction, destination, and protocols
with message flow configurations
- Service virtualization – Wraps existing systems or services with new service interfaces
- Protocol conversion – Bridges different protocols, e.g. JMS to HTTP
- Support for enterprise integration patterns (EIP) – EIP is the defector standard for enterprise
integration (http://www.eaipatterns.com/)
- Quality of service – Applying security, throttling, and caching
- Connecting to legacy and proprietary systems – Business adapter including SAP, FIX, HL7
- Connectors to cloud services and APIs – Salesforce, Twitter, PayPal, and more
- Configuration driven – Most functionalities are driven by configuration, but not code
- Extensibility – There are extension points that can be used to integrate with any custom
protocol or a proprietary system
With the advent of ESBs, most EAI vendors simply rebrand their EAI solutions to an ESB solution. Yet,
there are a few ESB implementations that have been built from scratch as lightweight, standard-based,
and interoperable ESBs.
4.1 Enterprise Application Integration (EAI)
The use of SOA as an architectural style and ESB as the infrastructure for implementation has been a
great success for most organizations. In recent times, most organizations have solved their integration
problems by leveraging ESB as the backbone of their IT systems. In addition, the way an organization
implements IT solutions was revolutionized and business functionalities are implemented as services
rather than applications. Hence, in general, using an SOA/ESB for enterprise integration needs is a
good move.
However, the enterprise IT space has seen some drastic changes; the four disruptive forces of
enterprise IT (i.e. mobile, data, social networking, and cloud) have prompted organizations to rethink
and redesign enterprise IT solutions to cater to modern enterprise needs. Just the SOA/ESB approach
cannot facilitate all of these new enterprise IT requirements.
SOA and ESB are primarily designed for internal interactions. Given the innate complexity of SOA/
Web services and ESB, the cost of cross boundary business couldn’t be significantly reduced. Owing
to limitations, such as complex service contracts, non-mobile friendly data formats, inability to carry
out frequent iterations and not able to support service versioning, accessibility limitations, and the
absence of monitoring and analyzing, the modern enterprise needed enterprise IT solutions that
looked beyond SOA and ESB.
4.2 Exposing Business Functionalities via Managed APIs
The disruptive forces of mobile, data, cloud, and social networking have propelled the modern
enterprise to a new level. Most enterprises found that their business functionalities are accessed
mainly via mobile apps than the regular web portal. The proliferation of apps motivates enterprises
to expose business functionalities in a more convenient way than services to enable mobile
access. Hence, enterprises need a simple, secured, and managed approach of exposing business
functionalities.
As a result of sheer demand, API management has emerged as a way of exposing business
functionalities in a managed, accessible, monitored, and adaptive way.
APIs in general are the business functionalities offered over the network on top of standard protocols
(mostly HTTP), with well-defined, but loose contract and with simple message formats.
However, it doesn't solve all the SOA/ESB limitations. Still, you can leverage APIs to overcome SOA/ESB
limitations by converting APIs to ‘managed APIs'.
Managed APIs are publicly advertised and enables users to subscribe and access the API (self-service).
They also support versioning and service-level agreements and can be secured and authorized to
ensure protection. Moreover, managed APIs can be monitored and monetized, thereby offering
further benefits to the enterprise in terms of enhancing business processes and increasing revenue.
4.3 SOA, Integration, and API Management: The Close Cousins
Enterprises can leverage the power of its integration platform and the power of API management by
combining SOA and API. In this regard, there are some key elements that should be noted.
- APIs cannot replace integration: There is a myth that APIs can replace integration, but APIs
primarily expose business functionality in a managed fashion; in other words, it does not
address any problems when dealing with disparate systems and technologies.
- Integration of internal services, systems, data and cloud APIs: To expose a business
functionality as an API, any organization would need to have an underlying implementation of
a service and the plumbing between all the required systems and services. For this, you would
need an integration backbone, such as an ESB, and the API management layer would sit on top
of it.
- Unable to mangle SOA for API management needs: Some vendors try to mangle SOA
frameworks to support API management. SOA is not designed to meet the requirements of API
management and often ends up with disconnected and unstable frameworks.
So, how do you build real-world enterprise IT solutions by leveraging ESB and API management? That’s
where the API façade pattern comes into play.
API Façade Pattern
The use of SOA, integration, and API management in combination to realize business requirements is
a very common technical requirement for most organizations. API façade is the standard pattern that
addresses those requirements. It's a simple, but extremely useful pattern as it exposes a business
functionality without the underlying technical complexities.
Figure 5: API façade: A Simple Interface to a Complex System
5. The Future of Integration Middleware
The integration landscape is looming over a storm that’s driven by the new forces of enterprise IT;
cloud, mobile, big data and social. Enterprises are moving beyond the traditional means of integration
and there is a proliferation of enabling services through social and mobile platforms and offloading
some part of the on-premise business to the cloud through software-as-a-service (SaaS) solutions.
Therefore, organizations need an integration infrastructure that can support current and imminent
integration needs. Hybrid integration platforms are a new way to connect cloud based, mobile and on
-premise systems. It involves a mix of on-premise, cloud, B2B, social, and mobile integration scenarios
of varying complexity, and only a suitable combination of on-premise and cloud-based integration
platforms can fulfill such complex requirements. A hybrid integration platform is a federation of two
major technologies.
-
SOA/ESB based, on-premise integration platform: This is a base set of features that include
communications, mediation, orchestration, transformation, QoS, security, monitoring,
administration, and management.
-
. Integration platform as a service (iPaaS): iPaaS is a cloud hosted integration platform designed
to consume cloud resources exposed as services. iPaaS is realized as a suite of cloud services
aimed at addressing a wide range of cloud, B2B, and on-premise integration and governance
scenarios.
As future demands become increasingly oriented toward iPaaS-based solutions, based on the
complexity and the type of integration scenario, iPaaS has been developed into different categories
of integration. There are a few major areas of iPaaS that can be identified in future hybrid integration
middleware solutions. Thus, organizations need a comprehensive integration platform to meet today’s
modern enterprise integration requirements. The WSO2 integration platform has helped many
enterprises meet their integration requirements.
6. WSO2 Integration Platform In Practice
In the modern enterprise IT world, most organizations want to convert their business to one that’s
connected. A ‘connected business’ allows customers, partners, employees, internal, and external
systems, as well as any other parties to deeply engage and smoothly collaborate with the enterprise.
The cost of transactions between the enterprises also reduces significantly, while the enterprise, its
customers, and partners gain a competitive advantage over conventional enterprises. The WSO2
middleware platform enables companies to build a connected business by providing comprehensive,
data to screen open source products that span the entire breadth of SOA, yet remain lean, simple to
use, and are cost effective.
6.1 Building an Integrated Business with the WSO2 Platform
The foundation of building a connected business is to have a solid integration backbone to build an
internally and externally integrated business. The WSO2 integration platform has the capabilities for
an enterprise to achieve this; the platform has a logical categorization of WSO2 middleware products
that are used in the enterprise integration space, as illustrated in Figure 5.
Figure 6: Building an Internally and Externally Integrated Business with WSO2 Integration Platform
WSO2 Enterprise Service Bus (WSO2 ESB) is the main integration backbone of the WSO2 platform,
which can be used to integrate anything with everything ranging from web services and RESTful
services to complex SAP-based integration scenarios.
WSO2 Data Services Server (WSO2 DSS) is primarily used to dynamically wrap existing heterogeneous
and disparate data stores with a service layer. WSO2 DSS can integrate data sources (RDBMS or
NoSQL), create composite data views, and host data services. With WSO2 DSS, you can completely
decouple your data stores from internal and external consumers of those data stores.
WSO2 Business Process Server (WSO2 BPS) enables developers to easily deploy business processes
written using the WS-BPEL standard and also serves as the business process management and hosting
environment. In addition, it supports human tasks where the business processes that need human
interaction are required. In contrast to WSO2 ESB, WSO2 BPS is designed for stateful and long-running
business processes.
WSO2 Message Broker (WSO2 MB) enables applications to exchange communications asynchronously
or publish messages for timely access by many subscribers. WSO2 MB supports JMS and AMQP
standards for persistence messaging and can be used in scenarios where guaranteed message
delivery is vital.
In addition to the above-mentioned products, WSO2 Application Server (used to host web services)
and WSO2 Governance Registry(registry and repository and SOA governance) are also used as
supporting products in this space.
delivery is vital.
6.2 Using API Management on Top of the Integration Layer
Once the functionalities are decomposed into services and they are externally and internally
integrated, you need to expose those functionalities as managed APIs to your customers, partners,
and internal parties, among others. For this, you can use the API façade pattern to design the solution
and WSO2 API Manager can be incorporated as shown in Figure 6.
Figure 7: Using an API Management Layer to Expose Business Functionalities on Top of the Integration Layer
The API management layer comprises the API publisher where the organization can publish its
APIs; the API store to expose the APIs to internal and external application developers; and the API
gateway, which is capable of serving the the actual API traffic from the API consumers.
WSO2 Identity Server is used to support authentication and authorization, and WSO2 Business
Activity Monitor or WSO2 Complex Event Processor are used for monitoring and analytics.
6.3 Hybrid Integration with WSO2
WSO2 offers its iPaaS solutions in two main streams. The ESB capabilities that we discussed in the
previous sections are offered in the form of ESB as a Service offering in WSO2 Integration Cloud. In
addition to ESB as a service, the pre-built integration scenarios are available as ‘Integration Templates.’
WSO2 iPaaS offerings use ESB behind the scene in all integration scenarios and extensively leverage
WSO2 ESB 'Connectors' (https://storepreview.wso2.com/store/) to intetegrae with cloud APIs.
7. Conclusion
In keeping pace with the evolution of enterprise integration, enterprises have to rely on innovative
solutions to be competitive. Mobile, data, cloud, and social networking too have created a greater
need for modern enterprises to seek cutting-edge and comprehensive solutions to address their
enterprise integration requirements. Organizations today require integration infrastructure that
can seamlessly connect cloud-based, mobile, and on-premise systems. The key to achieving these is
to adopt a solid integration platform to build an internally and externally integrated business that’s
future proof and able to adapt to any new requirement. The WSO2 integration platform’s logical
categorization of WSO2 middleware products enables enterprises to meet these needs in a costeffective
and timely manner. WSO2's single code-based platform also enables enterprises to leverage
the advantages of combining SOA and API management.