Media Firm Leverages WSO2 Capabilities To Seamlessly Integrate and Deliver Bundled Services To Subscribers
A US-based electronic media company that delivers financial information and news stories to subscribers worldwide was looking to put in place a media platform that will help the wire service bundle new published articles and deliver these to subscribers. The company was also looking to expose its suite of media assets to subscribers as well as offer these to external clients via an online feed. To be able to meet all of these requirements, the company needed to deploy a lean, simple, and easy-to-use integration platform and expose these capabilities via managed APIs.
Business Requirements - An Overview
- Syndicate new articles - The company’s many publications are being subscribed to by numerous publishers; the existing system has a set of pre-defined criteria while there’s also potential for user-defined criteria to be incorporated. Subscriptions are added according to criteria and these are then queried from a scheduled task to see if there are any new articles that match the criteria. Thereafter, the new articles are bundled and dispatched to the respective subscribers.
- Managed APIs - The firm has many media assets that would need to be exposed as REST, thereby enabling a user to search and retrieve various assets. Moreover, these managed APIs must be able to specify rules and policies to control how the API behaves.
- RSS feed - These above-mentioned media assets are provided to external clients in the form of RSS feeds;therefore, these need to be managed.
Why WSO2's Integration Platform
The WSO2 integration platform comprises all the capabilities that enable enterprises to engage and collaborate with its customers, partners, employees, and internal and external systems. The platform has a logical categorization of WSO2 middleware products that are used in the enterprise integration space:
WSO2 Enterprise Service Bus (WSO2 ESB) is the main integration backbone of the WSO2 platform, which can be used to integrate anything with everything ranging from web services and RESTful services to complex SAP-based integration scenarios.
WSO2 Data Services Server (WSO2 DSS) is primarily used to dynamically wrap existing heterogeneous and disparate data stores with a service layer. WSO2 DSS can integrate data sources (RDBMS or NoSQL), create composite data views, and host data services. With WSO2 DSS, you can completely decouple your data stores from internal and external consumers of those data stores.
WSO2 Business Process Server (WSO2 BPS) enables developers to easily deploy business processes written using the WS-BPEL standard and also serves as the business process management and hosting environment. In addition, it supports human tasks where the business processes that need human interaction are required. In contrast to WSO2 ESB, WSO2 BPS is designed for stateful and long-running business processes.
WSO2 Message Broker (WSO2 MB) enables applications to exchange communications asynchronously or publish messages for timely access by many subscribers. WSO2 MB supports JMS and AMQP standards for persistence messaging and can be used in scenarios where guaranteed message delivery is vital.
In addition to the above-mentioned products, WSO2 Application Server ((used to host web services) and WSO2 Governance Registry (registry and repository and SOA governance) can also be used as supporting products in this space.
Once the business functionalities are fully integrated, an API management layer would need to be added on top of the integration layer to expose those functions as managed APIs to customers, partners, or internal parties. WSO2 API Manager comprises an API publisher where the organization can publish its APIs; an API store to expose the APIs to internal and external application developers; and an API gateway, which is capable of serving the actual API traffic from API consumers.
The componentized nature of the WSO2 platform also enables the incorporation of WSO2 Identity Server to support authentication and authorization, and the WSO2 analytics platform for data monitoring and analytics.
Refer to 'The Evolution of Integration: A Comprehensive Platform for a Connected Business' white paper to learn more about our integration capabilities .
For this use case, we've used a combination of WSO2 ESB and WSO2 DSS and added API management capabilities on top of the integration layer; production is across two data centers as illustrated in Figure 1.
Figure 1: Solution Architecture
High availability was one of the customer’s stringent requirements, i.e. the system would need to be up and running at any given time. Hence, for each product, there will be multiple clusters in two geographically isolated data centers. Therefore, while having high availability with the two separate data centers, each data center itself is highly available with servers being clustered. As a result, eventually, starting from each site, there will not be a single point of failure.
The main entry point to the system is through a global load balancer, which routes messages to both data centers; WSO2 Enterprise Service Bus and WSO2 API Manager instances act as the entry points to each data center. WSO2 Data Services Server instances are internally used for data access operations required for the scenarios. WSO2 ESB is connected with a message broker to ensure all its messaging operations are executed in a reliable manner.
Scenario 1 - Content Syndication
In the content syndication use case, as illustrated in Figure 2, users join the system, register certain criteria, and subscribe to these. When certain content that matches those criteria are available, the content is sent to the subscribers via specified delivery options, such as an SFTP transfer, HTTP endpoint, or an e-mail message.
Subscriptions and criteria are mapped and stored using a virtualized data view created using data services. The data services here connect to an RDBMS data store and exposes a set of web services to the ESB to carry out the required persistence and lookup tasks.
New content will always come in to the system. Therefore, rather than evaluating the criteria and the subscriptions for each and every event, this operation will take place in a batch mode by using a scheduled task, which in turn will check the content and the subscriptions from time to time. This task will evaluate all the criteria, compile a set of target packaging formats (e.g. zip and tar.gz), and prepare them to be dispatched to subscribers. By meeting one of the main requirements of high availability, the scheduled tasks used here are fully coordinated, and support high availability/failover semantics.
Figure 2: Content Syndication
Scenario 2 - Managed APIs
There was a requirement to expose the content syndication services itself as managed APIs. With WSO2 API Manager the company will expose its back-end content syndication services as managed RESTful APIs. The standard throttling and access control mechanisms are built into WSO2 API Manager and any additional user-defined policies can be implemented using custom API handlers.
Scenario 3 - RSS Feed Generation
An RSS feed was required to be generated out of the company's existing content search API. Therefore, this RSS feed will expose data that's compiled in a specific format and will be accessible to the public. This operation was implemented using WSO2 Enterprise Service Bus capabilities where necessary checks, filtering, and transformation are carried out and exposed as an HTTP resource in RSS format.
Post deployment, the company has been able to seamlessly integrate its bundled services and offer these to its customers via managed APIs. WSO2’s specialized middleware has provided the company with out-of-the-box features required for their specific use cases, thus alleviating the need to reimplement tried and tested technology. Another key milestone achieved is that the company now does not have to maintain in-house applications implemented to carry out these scenarios, which were complex and difficult to manage prior to the WSO2 deployment. Instead, the company now has an easy-to-manage setup that includes unique features, such as WSO2 ESB, WSO2 DSS - solutions that can be quickly implemented, debugged, and deployed.