Introducing Choreo API Chat: Transforming Your API Interactions with AI

We're delighted to unveil Choreo API Chat, Choreo's innovative approach to interacting with APIs, powered by Generative Pre-trained Transformer (GPT) 3.5 large language model (LLM) from Azure OpenAI. This allows users to try out APIs running or proxied via Choreo using natural language, serving as a complementary tool to traditional methods. It offers a convenient way to quickly try out endpoints without delving deeply into API specifications or crafting intricate JSON payloads.

At the heart of this feature is our Choreo API Chat AI agent (powered by the Ballerina Agent Framework), specifically designed and optimized to interact with APIs using natural language. AI agent interprets the OpenAPI specification of your API, allowing it to recognize available resources and the expected data types. Utilizing the computational capabilities of an LLM, the API Chat agent processes your natural language input to determine both the best combination of API resources to answer your query and the data to send to and then invoke all the API resources.

Step into a new era of interacting with APIs with Choreo API Chat!

Key Benefits

Ease of Invoking APIs

No more delving into OpenAPI specifications to craft specific test messages. Simply convey your intent to the AI, and it will generate meaningful messages and handle the communication with your API.

Zero Coding Needed 

Choreo API Chat seamlessly manages complex scenarios involving multiple resource calls in various combinations, making API interactions a breeze.

How to Use Choreo API Chat

APIs can have a wide range of resources. Depending on the scenario, you might need to invoke a single resource, engage with a subset of resources, or even access all of them. Regardless of the requirement, Choreo API Chat can determine which combination of calls must be made for your query and do it for you.

Sample Queries

When you select API Chat, now accessible under "Test" in the left navigation menu of the Choreo Console, you'll be presented with sample queries specifically designed for your API. These queries help illuminate the variety of interactions that are possible with your API. This is illustrated in Figure 1.

Figure 1: Sample queries

Invoking All Resources

Sometimes, you may wish to try out all the resources within an API without diving into specific scenarios. With Choreo API Chat, achieving this is straightforward. You can either type "Invoke all resources" and then click on "execute", or select the sample query labeled "Invoke all resources". Witness this feature in Figure 2 below, demonstrated on a renowned PetStore API.

Figure 2: Invoking all resources in the Petstore API

Invoking a Single Resource

At times, the emphasis is solely on a specific resource. Using a straightforward natural language query, you can invoke an individual resource, seamlessly facilitating the desired interaction. The process is as intuitive as having a conversation, as shown in Figures 3 and 4.

Figure 3: Executing 'List Sold Pets' against the Petstore API

Figure 4: Executing a sample query on the Petstore API

Multi-resource Interactions

Handling queries that span multiple resources can be challenging. However, with Choreo API Chat, navigating complex scenarios becomes intuitive. In the case illustrated in Figure 5, watch as several resources are engaged and see how a comprehensive query is executed precisely.

Figure 5: Executing a query involving multiple resources

In each scenario, the intelligence of the API Chat agent stands out. Using natural language input and the API's OpenAPI specification, the agent, with the assistance of the Large Language Model (LLM), determines which API resource to invoke and the corresponding message to send. Upon making this determination, it invokes the resource. The outcome of this invocation guides the subsequent resource to be invoked, and this process repeats until the completion of the task. The API interaction appears seamless on the outside, yet a sophisticated artificial intelligence system drives it.

The Accuracy of Choreo API Chat

Choreo API Chat offers a novel approach to interacting with APIs. Its efficiency and natural language processing attributes make it a standout in this field. However, like all sophisticated technologies, managing expectations is essential.

The LLM Factor

While Choreo API Chat has been optimized for invoking APIs, it does, at its core, leverage the abilities of LLM. It might occasionally exhibit behaviors akin to LLM, leading to unexpected outcomes.

Re-run for Assurance

If you stumble upon a result that doesn't align with your expectations, we recommend a simple trick: re-run the same query. Given the dynamic nature of LLM-based systems, a second attempt may provide a more satisfactory outcome.

What You Give is What You Get: The OpenAPI Specification

The quality of your OpenAPI specification plays a pivotal role in determining the accuracy of Choreo API Chat's results. While it is designed to identify and mend gaps in the specification, the adage "garbage in, garbage out" holds some truth. Your API must have a well-crafted specification with semantically meaningful names and as much documentation as possible to get the most from the AI.

The Way Forward

Here's a brief glance at our roadmap:

• Precision Tuning: To improve Choreo API Chat’s performance (both in accuracy and speed), we are continuously improving our architecture and the overall developer experience. Your feedback is very helpful in this regard!

• Supporting GraphQL: Currently, Choreo API Chat only supports Rest APIs, but we are working on adding GraphQL support.

• VSCode Integration: We are working on offering WSO2 API Chat as a plugin for VSCode so any API developer can use it irrespective of their APIs being on Choreo.

• Action Bots: We are working on extending Choreo API Chat's capabilities to facilitate action bots that can combine and interact with multiple APIs. 

• Pluggable LLMs: We currently support Azure OpenAI’s GPT 3.5 as the LLM that Choreo API Chat uses. We are adding support for other LLMs, including open source ones.

This approach seeks to transform API interactions by leveraging AI. We’re committed to enhancing its accuracy, expanding its capabilities, and ensuring it becomes a vital tool in this domain. For a detailed guide, see the Choreo API Chat documentation.

We value your input! Join WSO2’s Discord server, and feel free to share your feedback about Choreo API Chat in our Discord today.