How to Get Over the Serverless Integration Hump

Image credits: Mikk Tõnissoo on Unsplash

Most of us know by now how developers can rely on a serverless platform to deploy applications at will in the cloud without having to worry about configuring or managing hardware, operating systems or servers. With this potential jump in the freedom that allows developers to spend more time creating and writing code, an often overlooked potential stumbling block are issues affecting integration.

We have taken this subject by researching the effects and outcomes of serverless integration. Based on our findings, we concluded that the difference between building apps and programming applications that mainly manage integrations is disappearing fast. However, before that major shift happens, the serverless environment will need an integration language to support those use cases.

In this post, we discuss why organizations will need to adopt a programming language specific to integrating IT environments with serverless platforms.

Programming Future Applications

The difference between building apps and building integrations is fast disappearing. The rise of the cloud, APIs and serverless architectures are making app development much more like integration, effectively making integrations ubiquitous. Applications are composed of APIs and the logic required to write those applications is similar to writing integrations. Consequently, the need for integration-like technology will be much more commonplace in future systems. This development will significantly widen the available market for integrations while driving existing programming languages and platforms to provide first-class support for writing integrations.

We can program simple integrations using programming languages, such as Java, Go and Python. However, complex integrations in serverless architectures also inherit most of the problems that occur in enterprise integration settings. For example, those integrations need to be built by composing APIs from different vendors, which leads to timing, data format, and context mismatches. Using a programming language, such as Java, to do so requires a great deal of detailed code. Furthermore, with the wide adoption of microservices and APIs, the number of services involved in integration will likely rise, increasing the complexity of integrations and services alike. To handle such scenarios, enterprise service bus (ESB) vendors, such as WSO2, Mulesoft, and IBM, have provided integration languages. It is likely that similar tools are required for integrations in the cloud as well.

Historically, early integrations were written using classic programming languages, such as COBOL, C, Java, and C#. However, most integration use cases involve communicating between services, APIs, data sinks and data sources and then matching their inputs, outputs, timing and control flow. Classic programming languages do not have first-class constructs for service invocations or deal with data formats such as XML or JSON. Hence, developers need to re-implement the logic from first principles, which leads to code that is verbose, complicated to write and hard to maintain. To address this challenge, on-premises integration vendors have supported domain-specific languages for programming integrations.

Currently, serverless platform users rely on languages, such as Java, to implement their integration use cases. However, as more complex use cases move to serverless architectures, we are likely to rediscover the limitations we faced with those languages years ago. Therefore, the lack of higher languages supporting integrations continues to be a challenge faced by serverless computing.

This can be solved using several means:

• Users can run integration middleware within the infrastructure as a service (IaaS) solution and use it to integrate serverless applications.
• Cloud providers can support integration middleware as one of their platform as a service (PaaS) services.
• Cloud providers can support integration languages - e.g., Ballerina or domain-specific languages (DSLs) - as languages supported by a PaaS.
• Solution providers can expand visual programming support for general integration.

Conclusion

While exploring the impact of serverless on integrations, we concluded that the difference between building apps and building integrations is fast disappearing and the serverless environment will need an integration language to support those use cases. We believe this observation will help organizations in planning the long-term strategy for their systems.

We found that serverless platforms that require significant integration support have a significant impact demonstrating that these are feasible. Assuming that serverless platforms will support an integration language, we expect to see many new applications adopting serverless architectures in the next three years. Since these applications use APIs and interact with external systems, as applications move to the cloud, integration use cases will move to the cloud with them. The full report of our findings can be found here.

This blog first appeared in The New Stack and published here with permission.