All posts by Ken Oestreich

WSO2 API Management Strategy: Industry Observations and Implications

Recently we at WSO2 were asked by a leading analyst to outline our vision and strategy for the future of API management. We felt that our response captured much of our current and planned execution, so much so that we felt we needed to share it. Our culture at WSO2 has always been one of transparency, and in the past, we’ve even shared our financials.

Following are some of our positions on API management and additional market insights, as well as our vision of the composable enterprise. Stay tuned for additional strategy-related posts that dive more deeply into our technology “big bets” and direction.

How digital transformation is changing the landscape of APIs and digital connections

Current IT trends show that over the next few years, enterprises will find they need to deal with more than 1 trillion programmable endpoints and APIs. These will consist of traditional application APIs, data APIs, data streams, software component APIs, microservices, sensors, and IoT inputs as well. Indeed, everything may become an API.

Knowledge workers know this, and will want/need access to all these APIs/endpoints whether it’s only to create a basic SaaS-to-SaaS connection, or to create a more complex integration. Therefore, over the next 2 to 5 years, we expect that tools and processes will necessarily evolve to address this level of scale and complexity.

Additionally, infrastructures to support this huge quantity of endpoints will gravitate toward those optimized for microservices and serverless underpinnings. From a development perspective, current low-code integration approaches that involve centralized IT orgs and/or waterfall style processes simply will not scale. As a result, architectures will necessarily tend toward more decentralized, cell-based approaches underpinned by microservices and serverless.

With the trend toward trillions of endpoints, WSO2 believes much of what is today considered part of the “development” organizations will evolve to include API integration. The trend will be particularly strong where APIs serve as the core of digital apps and applications that rely on Internet of Things (IoT) data and artificial intelligence (AI). This is at the core of the disruption WSO2 sees in the coming years: that IT organizations tend less toward “development”, and more toward being “API integrators.” We call this new disruptive IT phase the composable enterprise, which will be fueled by the explosive availability and use of APIs and programmable endpoints.

The future of digital connections across enterprise boundaries

WSO2’s position is that API ecosystems across enterprises will expand as today’s software disaggregation (componentization) trends continue. Thus the composable enterprise will become a combination of both internal and external API-based services, each front-ended by private and/or public APIs. This API diversity—and dynamism—will inherently require hybrid API integration capabilities and distributed (rather than centralized) forms of management and governance.

To accomplish this, we see the use of distributed integration technologies, such as microgateways and micro ESBs, which necessarily operate in a decentralized fashion, bridging services from different sources, vendors, and enterprises.

From a business perspective, WSO2 sees ever-tighter service integrations across enterprises, suppliers, partners, and customers—all underpinned by API integration technologies. IT departments will become the “services supply chain managers.”

A perfect example is WSO2 customer Wells Fargo, which has successfully front-ended its organizations and systems with public APIs and gateways to accelerate new product and service delivery, as well as speed integrations with business partners. This form of API marketplace is being adopted by digitally driven organizations that are encouraging partners, suppliers, and even customers, to work more closely with their offerings.

Enter: the composable enterprise

The WSO2 vision of the composable enterprise does not imply a purely internal IT model, but rather an approach that spans the enterprise’s complete external service ecosystem as well.

The notion of the composable enterprise will involve closer, more secure, and more real-time digital interactions between vendors, suppliers, and customers—as well as for internal integrations. API-based interactions will also result in more rapid product and service innovation among all parties, creating new forms of value for customers, partners, and internal business units alike. Already, multiple forms of storefronts, macro-gateways, and monetization models are arising where enterprises are brokering their internal services for use by external entities.

Today, WSO2 customers are pursuing this vision. Wells Fargo, BNY Mellon, and StubHub are just three of many enterprises that are publishing their APIs, as well as basing their internal architectures on disaggregated components front-ended with APIs, gateways, etc.

Indeed, many leading companies are already basing the bulk of their revenue on the API economy, capitalizing on the business wave highlighted in by the Harvard Business Review back in 2015:

“…Salesforce.com generates 50% of its revenue through APIs, Expedia.com generates 90%, and eBay, 60%. Salesforce.com has a marketplace (AppExchange) for apps created by its partners that work on its platform; they now number more than 300. Expedia’s APIs allow people using third-party websites to tap its functionality in order to book flights, cars, and hotels. And APIs allow eBay to list its auctions on other websites, get bidder information about sold items, collect feedback on transactions, and list new items for sale-all of which give additional exposure to eBay items and increase revenue.”

Future drivers and shapers of API management

WSO2 sees the major pressures driving the future of the API management space as grouped into two main categories: the market drivers led by the demand for API business and the technology shapers, led by vendors and innovators.

Drivers of API demand aren’t entirely new, but they have recently risen in their influence on IT behavior:

  1. The trillion endpoints future: the trend toward every digital asset becoming a programmable endpoint and causing IT to create strategies to access these assets.
  2. Digital business competitive pressures: forcing organizations to more quickly find ways to digitally interact with suppliers, partners, and customers.
  3. Knowledge worker information consumption: where organic demand for nearly every digital asset begins with line-of-business users looking for new data and conveniences.
  4. SaaS-to-SaaS app integration: a trend increasing exponentially where every new SaaS app or component is more valuable each time it’s integrated with another.
  5. Machine learning: with applications of ML forcing both data at rest and data streams to become accessible and front-ended with APIs.

Similarly, API management is being shaped by adjacent systems and technologies, quickly maturing the use (and re-use) of software endpoint components:

  1. Microservices and serverless technologies: these are (and will be) driving massive app disaggregation because of the abstractions and simplicities they create for software deployment, directly leading to a world of more broadly distributed micro APIs and microgateways.
  2. Cloud native dynamic systems: growing class of distributed and dynamically changing microservices will cause API discovery and surveillance to become dynamic as well.
  3. Configuration-based integration tools (e.g. ESBs) and code-based integration programming languages (e.g. Ballerina): because, “Software is eating the world,” every company is being forced to make software and agile integration to become a core competency. This creates a world where forms of API integration need to become as agile as developers and organizations want them to be.
  4. API security, access and governance: these requirements are leading to the native integration between integration, access Management, and API management.
  5. The advent of distributed cell-based architectures: these new architectures will allow for decentralized development, test and deployment, speeding integration activities across organizations.

Implications WSO2 sees for the future of API management solutions

  • Implications for architecture: there will be a growing shift toward cloud-native architectures and a need for decentralized composable units of architecture. Each composable unit is what WSO2 terms a “cell”. Cells are defined by, and interfaced through, APIs; are governed by micro- and macro-gateways; include embedded control planes like service meshes; and are developed by decentralized, independent teams.
  • Implications for development agility: with the need to develop and maintain an increasing number of connections across the enterprise, an organization’s ability to remain agile while supporting this expanded connectivity, faces pressures. WSO2’s vision is not only enabling organizations to make these connections, but to empower development teams, DevOps, and operations to increase their adaptive agility while doing integration. Integration teams must become integration agile, adopting the tools, organization, and processes similar to agile development.
  • Implications for tools: all API management and integration tools will need to involve some form of distributed technology, and all will necessarily evolve to be microservice and serverless friendly, i.e.:
    • Provide distributed forms of observability and security
    • Offer multiple control planes
    • Support service meshes
    • Support hybrid orchestration architectures

In closing…

Here at WSO2, we’re betting that all developer organizations will eventually have to adopt integration skills as well — especially as all digital assets become accessible and programmable.

We’re also anticipating the result will be the composable enterprise, shifting business onto a digital ecosystem. And to facilitate that, we’re building open source integration tools, integration agile methodologies, and even programming languages, to help digitally driven organizations achieve this future.

Stay tuned for more of our technical “big bets” in a future blog.

Announcing the WSO2 Serverless Solution

Most enterprises today looking for serverless solutions have few options without cloud lock-in. Remember that public serverless offerings will capture a customer’s data, lock out external event streams, and likely limit developer language choice. This lock-in hinders application migration, multi-cloud scaling, and the use of private cloud resources. A more palatable solution ought to allow organizations to tap serverless for disaggregated architectures, and allow them to utilize both public and private cloud resources, event models, and programming paradigms.

In response, customers today are mostly forced to use public serverless offerings from AWS (Lambda), MSFT, GOOG, etc., with limitations placed on the supported programming languages for each. Users are further locked-in because of the need to use adjacent proprietary services like the cloud’s storage services. And if a company wants to use an alternative, they’ll require considerable investment to manage.

Enter the WSO2 serverless solution

Today we’re introducing the WSO2 Serverless Solution, a private function hosting environment based on Apache OpenWhisk and Kubernetes. And it’s immediately available, though on a limited-access basis.

To develop the solution, WSO2 has been working with Rodric Rabbah and Perry Cheng, co-founders of CASM LLC and co-creators of Apache OpenWhisk. They bring in-depth knowledge on custom deployments and backend optimizations to the overall solution, and both continue to be active members of the OpenWhisk community.

The solution allows organizations to leverage their existing event sources and programming languages. Underlying the open source function platform, Apache OpenWhisk allows developers to plug existing event sources into the solution. It also allows developers to use their preferred programming language as a function runtime which will allow them to re-use most existing code, and allows users to define their own custom resource limits. These combine to provide greater overall agility to a serverless solution. And you’ll have freedom from cloud lock-in.

And the best part is that the WSO2 Serverless Solution is a private hosted platform managed by WSO2, so it ought to significantly reduce learning, set-up and maintenance overhead for DevOps teams.

A little more detail…

The serverless solution is fundamentally powered by Apache OpenWhisk and Kubernetes to allow IT orgs to provide a uniform, elastic, and secure platform for reactive, event-based, and batch workloads.

The Solution offers several unique capabilities:

  • Private function platform – powered by Apache OpenWhisk deployed on top of Kubernetes
  • Managed hosting environment – provided by WSO2, mapped to internal private resources and events, with customized elasticity.
  • Private, dedicated servers and operations – provides segregated tenancy
  • Support for any programming language – broader support than any single public cloud vendor
  • Leverage any existing event source – no matter where you deploy
  • Transparent computational elasticity – to support both short and long running computation
  • Guaranteed computational capacity – because it is a private function environment
  • Secure platform, plus service isolation, and encryption of data in motion
  • Local development environment – for developer teams
  • Dev tracing and operations of event-driven apps with logging, monitoring, and analytics

Why did we do this?

WSO2’s mission is to help digitally-driven organizations become integration-agile. And we do that with a platform of open-source Integration, API Management, Identity Management and related products. One core motive of ours (and of the overall open source model) is freedom from lock-in… So it stood to reason that if we wanted to simplify integration tasks, it would require simplifying deployment tasks too. So we developed this cloud-vendor-neutral deployment approach to complement our products.

Availability

As mentioned, the solution is immediately available on an early-access basis. Pricing is offered at a flat rate, on either a monthly or annual billing. For more information see the WSO2 Serverless Solution.

Four Warning Signs an Integration Wall is Approaching

The Integration and API Management markets are growing, expanding in both popularity and use. Enterprise App integration will surpass $33b by 2020, and other markets like iPaaS and Data Integration are growing at double-digit CAGRs. Enablers, such as containers and serverless technologies are only accelerating the move toward increased disaggregation of applications.

All seems rosy. And it mostly is.

But with the explosive growth of APIs and endpoints, traditional centralized tools like ESBs will become unsuitable, and simple low-code snap-together tools won’t scale to address the broader scope. We’re potentially about to hit an “integration wall” at high speed.

Consider the following four warning signs – some technical, some process – that I find are beginning to plague the integration market:

1. Waterfall Development for integration is hitting a wall.

Although most code development has shifted to an Agile Development model, the same can’t be said for Integration tools. As the quantity and diversity of endpoints increases, and as Integration projects become more diverse and complex, use of the waterfall model is beginning to slow down integration projects. And with a future where there will be billions of Integratable endpoints, it’s obvious that an Agile Development model for integration will need to become the norm.

2. Existing tools and programming languages aren’t optimized for Integration-at-scale.

Enterprises that currently use low-code, snap-together, centralized integration technologies (including iPaaS) will not be optimized for orchestrating, integrating, observing and governing the expansion of constantly-changing endpoints. Nor are traditional centralized approaches (think: EDI and older ESBs) prepared to handle increasing endpoint scale or diversity. Many of these existing tools are well-adapted for Line-of-Business or Citizen Integrators of relatively small-scale implementations but are far from well adapted for more complex integration-at-scale projects.

3. Current programming languages are not optimized for Integration.

With languages like Java/Spring or JavaScript/Node, developers can engineer flow, but must take responsibility for solving the hard problems of integration. With these languages, developers have to write their own integration logic or use bolt-on frameworks. Clearly a new programming paradigm will be needed long term.

4. The Exploding Endpoint Problem is very real.

As I referenced above, IT is ill-prepared to address the oncoming wave of service disaggregation, the diverse types of APIs, differing sources of service endpoints, challenges from Big Data, and multiple approaches to serverless IT. The industry is about to hit a scale and diversity wall. To wit,

  • 917 apps in use per enterprise (Netscope, 2016)
  • 893-1206 average cloud services used per employee (Kleiner Perkins, April 2017)
  • 19,000 APIs as-of January 2018 (Programmable Web, 2018)

And if you don’t believe those numbers, Matt Eastwood of IDC recently pointed out that the number of containerized services has expanding well beyond where VMs ever were. Yep, billions of programmable endpoints aren’t kid’s stuff.

Where does this leave us?

A new approach to addressing the future of integrating thousands-or millions-of endpoints could lie in a new programming language, Ballerina.

Ballerina is a simple programming language whose syntax and runtime have been optimized for the hard problems of integration. Its focus is integration – bringing concepts, ideas and tools of distributed system integration into the language. Based on the concepts of interactions within sequence diagrams, Ballerina has built-in support for common integration patterns and connectors, including distributed transactions, compensation and circuit breakers. And it supports JSON and XML, making it simple and effective to build robust integration across distributed network endpoints.

So, watch this space for future developments. And in the meantime, beware of the approaching wall.