Fast Innovation for New Integrations, Microservices, and APIs
- Chamin Dias
- Associate Lead - Marketing - WSO2
First published on The New Stack.
The rapid growth in the development and use of APIs shows no sign of slowing down, and the central role APIs play in digital business across industries continues to surge right along with that growth.
One recent survey of IT leaders found that 98% consider APIs mission critical, with 81% currently working with microservices and 18% planning to do so soon.
The exponential growth in APIs has been such that most software in use today either uses an API or is one. APIs and the integrations and microservices behind them have become the basic building blocks for the new digital channels to customers, employees, and partners.
The Challenges That Slow Down Innovation
And yet, you might have read that most digital business initiatives (several studies mention 70%) over the past decade have failed. Either these digital initiatives did not deliver all of the expected benefits — such as new sources of revenues, increased revenues per customers or increased profits — or they delivered none of them.
Two major reasons come up repeatedly about why organizational productivity decreases and time to market slows down as the number and complexity of APIs grow. The first is about people: There’s a shortage of developer and operations skills needed to manage modern, cloud native infrastructure running on top of relatively new technologies like Kubernetes.
The second is about processes: DevOps processes are slowed down by the increasing requirements for security, governance and coordinating changes across all those who use an API. As API usage grows, the complexity of managing hundreds or thousands of APIs can become overwhelming, leading to increased costs and security risks that can undermine business benefits. As development teams are forced to devote more time to administrative tasks and governance, productivity often suffers and release times stretch out, which slows down innovation and growth.
If you dig deeper into the root causes, most of the time it was because these organizations did not design, develop, and manage their APIs, microservices and integrations together in a way that allowed them to innovate fast and promote reuse. This is not easy because of:
- API development: API design, development and deployment requires a CI/CD pipeline that delivers in hours, which is hard.
- API reuse: APIs exist to be reused, in part to enable development in hours. If an API is not easy to use, discover, share, or change for any reason, its value disappears.
- API deployment: It is hard to combine Kubernetes, API management, integration, service development, security, and other technologies into an easy-to-use digital platform.
- API governance: Managing users, API life cycles and security can lead to very manual processes that add too much time, cost, and risk.
- API visibility: End-to-end visibility for monitoring, audit trails or developer collaboration is hard to manage across APIs, integrations and services that span many organizations.
As the total number of APIs that you develop or use grows into the hundreds or even thousands, managing all these APIs along with the integrations and microservices they use quickly becomes overwhelming.
10 Best Practices for Fast Digital Innovation
The good news is many companies have been able to innovate fast by relying on several common best practices. WSO2 has summarized these based on its experience with over 1,000 deployments across companies. There are more details on them here.
1. Focus on Digital Experience Engineering
You cannot buy a unique experience that attracts and retains customers. You must build it and continually improve it over time. We call this “digital experience engineering.” Your team, technologies and processes all need to be focused on building and improving that experience fast.
2. Become API-First
If you want API consumers (developers) to adopt your APIs, you need to treat APIs as products. Design each API from the outside in to improve the API consumer’s experience. (Don’t expose existing integrations or interfaces as APIs. That is an inside-out design).
3. Assign a Product Manager for Every External API
Every good product has a product manager. Assign a product manager for each external API who will be responsible for that API from cradle to grave.
4. Build API-Led Integration and Services
The only way you can develop an API in hours is if most of the work involves reusing existing components designed specifically for reuse during API development. API-led design means each API, service and integration should be reusable and loosely coupled. Most changes should be able to be made within a single component without affecting other components.
5. Offer a Digital Platform as a Shared Service
Have a core team provide a full, cloud native digital platform as a simple-to-use shared service. It should provide API and app dev teams with simplified DevOps and CI/CD tools that combine service development, integration, and API management.
6. Support Multiple Developer Frameworks and Tools
You need to support different types of developers and skill sets. While you should make it easy for developers to create new APIs, you also need to make it easy for others to reuse, integrate or compose APIs as well for broad adoption.
7. Provide Simplified DevOps as a Default Framework
While it is important to support different developer needs to maximize adoption, you should provide at least one preferred environment for developers that codifies all the best practices around the digital platform and that simplifies DevOps to help make the path to fast innovation easier.
8. Integrate a Shared Service Catalog into All Tools
Everything that needs to be reused needs to be easy to discover, learn, adopt, and change over time. That requires a developer portal at the very least. However, ideally you should also provide a developer marketplace within the development environment that allows for adding components and additional coding.
9. Build in Governance from Security to Life Cycle Management
Make security and governance processes part of a common DevOps process that is streamlined and automated as much as possible. If you build API discovery, API design, API security and API life cycle management into the development process, you will increase reuse, accelerate the development of new APIs, lower the amount of rework and not just cut out manual “bolted-on” post-development processes. This will help deliver APIs much faster.
10. Implement Cloud Native Services and CI/CD for Teams
There is no advantage to each team reinventing their own Docker/Kubernetes infrastructure or multistage CI/CD pipelines. Make it easy for them to develop, deploy, monitor, secure and manage APIs, integrations, and services together as a composite API or application. Abstract away the complexity of cloud native deployment wherever possible.
The Path to Fast Innovation
No digital journey is the same, or easy. But if you follow these rules, you have a much better chance of success. You can read the whitepaper that digs into them in more detail. There’s also a webinar explaining some of these best practices at length.