With emerging digital businesses that create novel products and services, digital transformation is key for both established and modern organizations. This paper will analyze three industry surveys conducted by independent and reputed establishments that explore the challenges faced when adopting a digital transformation strategy.
Based on the findings of the surveys and other empirical evidence, APIs and API management solutions were identified as a key component in any digital business. The paper will explore the varied aspects an organization must consider when evaluating an API management solution and will examine how WSO2 API Manager can be used to solve their digital transformation needs.
2. What is digital transformation?
Solis and Szymanski define digital transformation as "the realignment of or investment in new technology, business models, and processes to drive value for customers and employees and more effectively compete in an ever-changing digital economy"1.
It can be thought of as an extension of the vision laid out in software architecture paradigms such as service-oriented architecture (SOA) and IT governance models where the organizational desire for improved agility is a driver of adoption. Digital transformation is a cross functional and holistic approach that transcends both technological and operational boundaries of an organization and enables it to stay relevant in an increasingly competitive and customer-driven world.
The importance of including operational aspects when transforming were addressed in vendor-specific SOA governance models such as those done by Oracle and IBM2 and other models proposed through research like one done by Varadan et al3. Further, Joukhadar and Rabhi’s research involving a literature review and a consequential analysis by means of industry expert interviews found that managing organization change is a key area of importance apart from technological aspects such as process monitoring, evaluation and security2.
"Insight Platforms Accelerate Digital Transformation" a Forrester report, states the following about the importance of digital transformation, "Digital is putting pressure on your business. Almost two-thirds of decision-makers at enterprises say addressing rising customer expectations is a high or critical priority, while half report that creating a comprehensive strategy for implementing technologies critical to digital transformation is a high or critical business priority"4.
2.1 Drivers and consequences
Digital transformation is a relatively new concept, a view that we believe is reflected in "The 2016 State of Digital Transformation" report by Solis and Szymanski compiled through a survey done on 528 digital transformation leaders and strategists. The researchers found evolving customer behavior and growth opportunities to be the two most dominant adoption drivers with survey compliance of 55% and 53% respectively. The report identifies accelerating innovation, modernizing IT infrastructure and improving operational agility as top initiatives for organizations embracing digital transformation1.
Though changing customer behavior is cited as a key driver, based on the findings of Solis and Szymanski1, majority of practitioners appear to be facing difficulties while mapping out their customers’ journeys and proactively identifying and dealing with changing customer behavior.
When looking at the consequences of adoption, there appears to be empirical evidence to suggest improvements in market valuation, net profits and productivity. One study on the effects of data-driven decision making on organizational performance found a 50% increase in market valuation and a 6% increase in profits5. Another study on the organizational effects of API adoption reports a market value increase of 10% and a profit increase of 5%6.
Though these studies were not done to assess the direct consequences of digital transformation, since they are concerned with digitization of organizational assets inline with digital transformation they can be considered as valid pointers towards the benefits of adoption. On the other hand, Solis and Szymanski's direct inquiry into the consequences of digital transformation found 41% increase in market share and 37% increase in customer engagement through digital channels1.
Ascertaining organizational agility through digital transformation is a must to proactively adapt and navigate a customer behavior-driven reality. In their report, Solis and Szymanski find that the need to modernize the IT infrastructure with increased agility is one of the top items in the digital transformation agenda for many executives and senior leadership1. A part of the decision to do so may be influenced by the familiarity with similar endeavors in the past. Organizations that are familiar with transformation projects based on SOA may already be aware of the benefits of digitization and repercussions of stagnation. They may also have come to understand the challenges that projects of this nature inherently embody. But before we discuss these challenges, it’s important to be versed in a common denominator of such projects.
3. The role of APIs in digital transformation
Benzell, Lagarda and Alstyne capture the definition of APIs as follows; "An API or application programming interface is a set of routines, protocols, and tools that standardizes building software applications compatible with an associated program or database. APIs are code. They are also contracts"6.
For most organizations 'Application Programming Interfaces' mean exactly what the name implies: a means in which a functionality of an information system can be exposed to an external system. Modern day APIs have come to be the way they are through a process of technological evolution of the mechanism in which monolithic systems are integrated with one another, pushed by a desire for atomicity, compatibility and agility. A journey that has gone through technologies such as remote procedure calls, web services, REST and its latest incarnation, microservices.
An API or application programming interface is a set of routines, protocols, and tools that standardizes building software applications compatible with an associated program or database. APIs are code. They are also contracts
Banzell, Lagarda and Alstyne (2016) The Impact of APIs on Firm Performance
APIs provide a means of extrapolating additional value from existing data and systems. They allow organizations to transition from entities governed by centralized IT to open IT that promotes self-service. This transition can remove traditional IT bottlenecks faced when realigning businesses to changing customer demands. Further, APIs provide a means of obtaining agility without having to compromise on the ability to govern organizational data or usage behavior. All of these capabilities are of paramount importance in digital transformation efforts as identified in MuleSoft’s Connectivity Benchmark Report: The State of Digital Transformation and APIs, 20167, a concise report compiled based on the finding of a survey done on 802 information technology decision makers spanning multiple industries and geographical boundaries.
According to the report, 96% of participants are already engaged in digital transformation or are planning to in the near future. The report identifies the misalignment of business and IT, time constraints, legacy infrastructure and systems and integrating siloed apps and data as four of the most pressing challenges in implementation. 66% of the participants share the view that if these challenges to transformation are not addressed it would affect their organization's revenue negatively in a period of 6 months. The three strongest responses to this demand include building and managing APIs, modernizing legacy systems and moving to the cloud. These responses tally with the business needs that drive API strategy as identified in the survey, which were integrating new software with existing systems, creating more value from existing systems, and increasing speed and creation of self-service IT teams. Lastly, the survey states that 44% of participants identified APIs as a fundamental component of meeting digital transformation demands7.
In summation, MuleSoft's Connectivity Benchmark report identifies the need for swift digital transformation and the challenges it creates. More importantly, the relationship between the response strategies to the pressing need of digital transformation and the drivers of API strategy suggests the importance of APIs in achieving digital transformational goals.
Now let’s see how APIs when combined with API management solutions can tackle the challenges identified in the report.
3.1 Types of APIs
Benzell, Lagarda and Alstyne identify three classifications of APIs based on characteristics of the consuming party6. These being
- B2C APIs: interfaces designed to be consumed by an organization’s customers.
- B2B APIs: interfaces designed to be consumed by other organizations. As with electronic data interchange (EDI) systems the objective of such APIs are to provide a means of sharing business critical information with other inter-organizational parties in a reliable and secure manner.
- Internal APIs: interfaces designed to be consumed intra-organizationally.
Tech savvy organizations that have embarked on SOA-based digitization journeys are likely to be familiar with the last two varieties of APIs. APIs may also be categorized based on accessibility; whether they are open to the world outside the organization or not. Each type of API carries with it its own set of challenges to be met and requirements to be realized.
Exposing the right information to the right organizations or individuals can lead to new revenue streams, value addition or insights into market behavior.
A well thought out IT architecture consisting of internal APIs could provide organizations greater control over its assets and provide a scalable infrastructure on which usage behaviors can be managed and governing rights can be propagated. The flexibility brought on by the pluggable nature of an API-driven architecture could amount to greater investments down the line, as prospective investors may assess whether the organization is fit for change. The empirical evidence presented previously on the consequential increase of market valuation for adopters suggests this hypothesis. Furthermore, collecting and analyzing data from internal APIs for behavioral patterns can give new insights into the workings of organizations, making way for more proactive decision-making.
Exposing the right information to the right organizations or individuals can lead to new revenue streams, value addition or insights into market behavior. These information points can originate from any of the three types of APIs: internal, B2B or B2C. The value of external APIs may not be immediately apparent or predictable to their creators as the value creation lies externally in the hands of the organizations and the individuals who combine these APIs across disparate systems in unpredictable ways. If organizations choose to directly monetize APIs they should adopt a software product or service provider's mentality and build-up on competencies needed for such endeavors. Further, leveraging APIs for B2B information flow can work as a cost-effective replacement of electronic data interchange (EDI) systems6.
With APIs of this nature, value creation can be indirect and focused on business ecosystem enrichment. For example, an airline carrier exposing passenger travel data to accommodation providers or an agriculture equipment manufacturer exposing information about soil or crop to pesticide or seed manufacturers.
3.2 Consequences of API adoption
In Benzell, Lagarda and Alstyne’s study on the impact of APIs on organizational performance, the researchers compared the projected outcomes in terms of income and market value of 132 organizations before API adoption with the actual outcomes after adoption. The adoption and usage data was gathered over a period of three years where the composite number of calls per year was between 60 - 800 million. The research findings revealed a modest 3% increase in net profit and a 10% increase in market valuation for the API adopters.
A closer analysis of the findings reveals external APIs as the dominant driver of the increase in market valuation, while elucidating the misconception that B2C APIs would lead to sales growth. The researchers rationalize this finding by stating the following “B2C calls would relate not to additional revenue sources, but rather, lower cost ways to provide previous services.” Moreover, the research attributes adoption of internal APIs with organizational cost reductions6.
A closer analysis of the findings reveals external APIs as the dominant driver of the increase in market valuation, while elucidating the misconception that B2C APIs would lead to sales growth.
3.3 The need for API management and the anatomy of an API management solution
Organizations that have embarked on digitization journeys in the past, especially those involving APIs may be familiar with key requirements and challenges of adoption including
- Integrating systems over departments to expose composite interfaces.
- Aligning existing identity management systems to ascertain the desired governing rights and usage behavior for the newly created artifacts.
- Governing the life cycle of the artifacts through different stages of their existence.
In the case of B2B or B2C APIs, ensuring external parties adhere to organizational policies and facilitating the external value addition process may be some common areas of focus. In line with this view, Benzell, Lagarda and Alstyne discusses how a decision made by Jeff Bezos, the founder of Amazon, mandating the use of APIs for inter-organizational business flow drove the company's phenomenal growth, and put it in line to become the tech juggernaut it is today6. According to API Evangelist Kin Lane, some of the key technical challenges faced in Amazon’s journey were support for APIs over departments, concerns of security, throttling/quota policy conformance and discovery and lifecycle governance8.
- API Registry: a component responsible for various aspects of API artifact governance
- API Gateway: a component responsible for mediation aspects such as mediation time transformation, security and throttling
- Developer Portal: a component intended to be used as an interface between the API providers and the external parties who wish to consume it
API management solutions that address API design need by providing mechanisms for
- Message level protocol transformation, for example from legacy SOAP envelops to JSON payloads.
- Tackling the intricacies transformations pose such as defining counterpart REST based resources or contract generation (Swagger).
- Defining access control decisions such as throttling, quota or authorization attributes.
They may also provide means of putting in place mediation level data transformation or enrichment logic. However, when utilizing such capabilities, organizations should keep in mind that mediation capabilities of API management solutions are best suited for simple use cases and should not be thought of as a replacement for a strategically placed enterprise service bus backbone. Further, organizations should ensure that an adequate level of granularity is achieved when defining governance rights to maximize the likelihood of desired API usage behavior.
In terms of API discovery, API management solutions should provide mechanisms to foster usage by external parties. Most solutions address this requirement with a dedicated developer portal. Developer portals may provide external parties with the means of managing APIs by grouping them and allowing the creation of application-specific extractions. Developer portals should make the API adoption process as frictionless as possible, providing convenient mechanisms for discovery, evaluation, and acquisition. Therefore organizations will do well to opt for solutions that can provide capabilities such as Bring Your Own Identity (BYOID) and industry standard contract definitions such as Swagger and other tools to make the assessment process quick and painless. Organizations that wish to expose B2C or B2B interfaces for direct revenue generation may also look to solutions that provide monetization capabilities out-of-the-box. Another point of interest for such entities can be the facilities provided for community building.
As discussed throughout this paper, improving agility and responsiveness of organizations to changing customer behavior should lie at the heart of any digital transformation endeavor. Therefore a key ingredient of any such venture should be collecting the information from the data integration points. Since APIs have become the smallest denominator in such transformations, it is imperative that organizations pick an API management solution that provides the flexibility of drawing the analytical story specific to it. Organizations should focus on both data collection and analytics capabilities, as these facets are equally important in ascertaining the desired outcomes. These needs will be addressed in more detail later on.
Improving agility and responsiveness of organizations to changing customer behavior should lie at the heart of any digital transformation endeavour.
Modernizing IT infrastructure is an initiative that is often pursued by organizations1. They can benefit from opting for solutions that maximize (re)usage of existing infrastructure. In line with this goal, API management solutions that provide identity federation capabilities by using industry leading protocols such as OAuth and Security Assertion Markup Language (SAML) to provide granularized authorization such as XACML allows the reuse of existing identity management systems and infrastructure. They also provide organizations a means of addressing concerns of governance rights and allows them to escape from identity silos. Further, organizations may do well to look for solutions that support existing databases management systems or user management systems to minimize infrastructure wastage.
Organizations can evaluate solution capabilities in confidence by understanding the different technological constraints and requirements of each type of API discussed earlier, such as identity federation and provisioning concerns when dealing with B2B APIs or analytics and anomaly detection needs when dealing with external APIs.
As customer experience is cited as the foremost driver of digital transformation, organizations should hold analytical capabilities at the top of its list of priorities when evaluating API management solutions. As identified by other research done on the subject, analysis of data from integration points could provide insights into the customer journey, and changes and trends in customer behavior. The analytical story that needs to be designed is one that is unique to each organization, as discussed in operational and business aspects of digital transformation and SOA governance models. As such, what organizations should look for in terms of analytical capabilities of API management solutions is the feasibility and operational costs involved in realizing the identified analytical stories.
As customer experience is cited as the foremost driver of digital transformation, organizations should hold analytical capabilities at the top of its list of priorities when evaluating API management solutions.
As discussed previously, the demands of API management far exceeds those set out by its definition. In summation, some key points of interest for those who wish to use API management solutions in their digital transformation efforts are the
- Developer portal for value creation.
- Analytics capabilities.
- Flexibility provided to organizations in terms of defining governance rights and stories.
- Cost of extending to other areas of transformation such as integration and identity federation.
- Use of industry leading protocols and standards in the solution.
4. WSO2 API Manager
The subsequent paragraphs will examine how WSO2’s API management solution can be best utilized in practice to meet the demands of digital transformation.
WSO2 API Manager is a complete API management solution built around WSO2's open source middleware platform. WSO2 is trusted by industry leaders such as eBay, Cisco and StubHub for their business critical systems. Organizations evaluating API management solutions may feel rest assured by the technological and functional competency of the product. It was cited in The Forrester WaveTM: API Management Solutions, Q4 2016 report, an in-depth industry report evaluating 14 API management solutions. Apart from being a strong performer, the report also places WSO2 API Manager among the top three ranked in terms of the current offering category, covering criteria such as product architecture, deployment options, security integration, platform integration, API documentation, API quotas and rate limiting10.
The solution consists of five uniquely distinguishable components that can be separately deployed based on customer preference and expected runtime demands.
- Gateway: the entry point into the solution for all external invocations and the point of origin for mediation capabilities of the solution.
- Key Manager: provides identity needs for the solution such as access token generation and identity extension.
- Traffic Manager: provides throttling and rate limiting capabilities.
- Publisher: provides API developers the tools needed to design and publish APIs. Also provides administrators the tools to manage the solution as a whole.
- Developer Portal: also known as 'the store', is the hub for external API consumers to discover, explore, evaluate and adopt published API.
Figure 1: WSO2 API Manager Architecture
4.1 Meeting the analytical needs of digital transformation
As improving organizational responsiveness is a key driver in digital transformation, organizations should focus on digitization that enables analytical insights into customer behavior and demands.
WSO2 API Manager provides some analytical capabilities out-of-the-box, but it should be integrated with its analytics component to unlock the full capability. Integration is simple and straightforward. It provides real-time analytics with its battle-hardened stream processing engine 'Siddhi', trusted by industry disruptive behemoth Uber as its processor in their stream processing pipeline responsible for crunching 30+ billion messages per day11. Batch processing capabilities, including predictive analytics, are provided with the help of Apache Spark. In terms of analytical collection, the solution is configured out-of-the-box to emit analytical events from multiple critical operational points, encompassing the full API lifecycle from creation to invocation, spanning API designers and application developers. Some of the key operational points supported by default are the invocation request and response flow, rate limiting and throttle flow, invocation flow and developer application/subscription flow. These event streams are then configured to be summarized in multiple useful ways in real-time and through batch processing to produce a collection of valuable perspectives into the solution.
As the analytical story of an organization is one that is unique to it, organizations are likely to find the need to extend the analytical capabilities of the product. The extensions may be in the form of
- Summarizing available information in new and meaningful ways.
- Putting in new analytical data collection points.
- A combination of collection and summation extension.
- Extending the analytical framework itself to accommodate existing infrastructure.
As with any open source software, adopters of WSO2 API Manager stand to benefit from the ease of customizability and technological flexibility intrinsic to this type of software.
A typical extension of real-time summation capabilities may involve creating or editing execution plans using SiddhiQL (a simple querying language similar to SQL) and configuring output adaptors (a mechanism governing post process behavior). Whereas a typical extension of batch summation may involve creating or editing Spark SQL scripts and configuring Spark Jobs as appropriate.
By simply integrating WSO2 API Manager with its analytics component you can unlock its full analytics capabilities.
When extending the analytical collection capabilities, organizations should first identify the correct components of the solution that should emit the new data. Based on the component and the complexity of the data that needs to be collected, the best-suited means of data publication should be selected.
Simple mediation time data publication may be achieved through the use of an API mediation policy coupled with an Event Publisher Mediator. In the case of a complicated data collection requirement, implementers may benefit from defining the logic through code, utilizing mechanisms such as Mediation Handlers or Data Agents. Implementers would then need to create stream definitions, a logical representation of the data on the analyzer side and plug it to a summation method of their liking. Further, organizations may also plug in their own data publishers by implementing interfaces provided.
As with identifying summation requirements, organizations would do well to consult the expertise of data scientists to ensure the summarized data is interpreted correctly when acting on analytical insights.
4.2 Aligning other areas of the solution
When considering other functional areas of the product, organizations will do well to align aspects of API design and discovery in line with goals of digital transformation. WSO2 API Manager provides a set of policies governing throttling, mediation and some aspects of access behavior out-of-the-box. Aspects of API design may be carved to specific organizational needs through the definition of new policies. Moreover, administrators may define new throttle tiers or mediation flows to be used by API developers. They may even enforce global mediation decisions in line with organizational needs. As WSO2 API Manager provides interfaces to plug in and extend many critical points of operation, organizations may carve it as required.
The identity management capabilities of API Manager may be extended with little operational cost by integrating the solution with WSO2 Identity Server. This integration would allow organizations seamless identity federation capabilities over disparate identity silos and organizational boundaries, facilitating simplified access and adoption for external API adopters.
WSO2 API Manager may extend to other areas such as organizational structure and processes. The comprehensive user management capabilities of WSO2 API manager coupled with the business workflow capabilities supported out-of-the-box can be utilized to ensure departments and groups adhere to governance decisions. As with analytical capabilities, workflow capabilities of the solution can be extended to organizational needs at minimal operational cost.
In summation, some of the key competencies and advantages of WSO2 API Manager are, its functional competency which far exceed its competition, its advanced analytical capabilities, the low operational cost of extending or customizing the solutions behavior or capabilities and the assurance of a industry reputed provider.
Find more information on the WSO2 API Manager here: wso2.com/api-management.
Digital transformation is a journey modern organizations must embark on. The repercussions of stagnation can be dire. This white paper identifies API management as a key component of an organization’s digital transformation strategy, as it can address some of the toughest challenges met by those in transformational journeys. When evaluating API management solutions, the paper recommends focus be put on analytical capabilities and customizability and extensibility of the solutions in line with transformational needs that are specific to the organization.