Skip to content

LLM Proxies

LLM proxy management operations

Create a new LLM proxy

POST /llm-proxies

Code samples

curl -X POST https://localhost:9243/api/v0.9/llm-proxies \
  -H 'Authorization: Bearer {access_token}' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json' \
  -d @payload.json

Deploy a new LLM proxy configuration.

Payload

{
  "id": "wso2-con-assistant",
  "displayName": "WSO2 Con Assistant",
  "description": "Customer support assistant",
  "version": "v1.0",
  "projectId": "default-project",
  "context": "/openai",
  "vhost": "api.openai.com",
  "provider": {
    "id": "wso2-openai-provider",
    "auth": {
      "type": "api-key",
      "header": "X-API-Key",
      "value": "my-api-key-value"
    }
  },
  "additionalProviders": [
    {
      "id": "anthropic-provider",
      "as": "anthropic-upstream",
      "transformer": {
        "type": "openai-to-anthropic",
        "version": "v1",
        "params": {}
      }
    }
  ],
  "openapi": "openapi: 3.0.3\ninfo:\n  title: Proxy API\n  version: v1.0\npaths: {}\n",
  "globalPolicies": [
    {
      "executionCondition": "request.header.x-custom == 'enabled'",
      "name": "SET_HEADER",
      "params": {
        "key": "MyHeader",
        "value": "MyValue"
      },
      "version": "v1"
    }
  ],
  "operationPolicies": [
    {
      "name": "token-based-ratelimit",
      "version": "v1",
      "executionCondition": "string",
      "paths": [
        {
          "path": "/chat/completions",
          "methods": [
            "GET"
          ],
          "params": {}
        }
      ]
    }
  ],
  "policies": [
    {
      "name": "budgetControl",
      "version": "v1",
      "paths": [
        {
          "path": "/chat/completions",
          "methods": [
            "GET"
          ],
          "params": {}
        }
      ]
    }
  ],
  "security": {
    "enabled": true,
    "apiKey": {
      "enabled": true,
      "key": "X-API-Key",
      "valuePrefix": "Bearer",
      "in": "header"
    }
  },
  "associatedGateways": [
    {
      "id": "prod-eu"
    },
    {
      "id": "prod-us"
    }
  ]
}

Authentication

Parameters

Name In Type Required Description
body body LLMProxy true none

Example responses

201 Response

{
  "id": "wso2-con-assistant",
  "displayName": "WSO2 Con Assistant",
  "description": "Customer support assistant",
  "createdBy": "john.doe",
  "readOnly": false,
  "updatedBy": "john.doe",
  "version": "v1.0",
  "projectId": "default-project",
  "context": "/openai",
  "vhost": "api.openai.com",
  "provider": {
    "id": "wso2-openai-provider",
    "auth": {
      "type": "api-key",
      "header": "X-API-Key"
    }
  },
  "additionalProviders": [
    {
      "id": "anthropic-provider",
      "as": "anthropic-upstream",
      "transformer": {
        "type": "openai-to-anthropic",
        "version": "v1",
        "params": {}
      }
    }
  ],
  "openapi": "openapi: 3.0.3\ninfo:\n  title: Proxy API\n  version: v1.0\npaths: {}\n",
  "globalPolicies": [
    {
      "executionCondition": "request.header.x-custom == 'enabled'",
      "name": "SET_HEADER",
      "params": {
        "key": "MyHeader",
        "value": "MyValue"
      },
      "version": "v1"
    }
  ],
  "operationPolicies": [
    {
      "name": "token-based-ratelimit",
      "version": "v1",
      "executionCondition": "string",
      "paths": [
        {
          "path": "/chat/completions",
          "methods": [
            "GET"
          ],
          "params": {}
        }
      ]
    }
  ],
  "policies": [
    {
      "name": "budgetControl",
      "version": "v1",
      "paths": [
        {
          "path": "/chat/completions",
          "methods": [
            "GET"
          ],
          "params": {}
        }
      ]
    }
  ],
  "security": {
    "enabled": true,
    "apiKey": {
      "enabled": true,
      "key": "X-API-Key",
      "valuePrefix": "Bearer",
      "in": "header"
    }
  },
  "associatedGateways": [
    {
      "id": "prod-eu"
    },
    {
      "id": "prod-us"
    }
  ],
  "createdAt": "2023-10-12T10:30:00Z",
  "updatedAt": "2023-10-12T10:30:00Z"
}

400 Response

{
  "status": "error",
  "code": "VALIDATION_FAILED",
  "message": "The request failed validation.",
  "errors": [
    {
      "field": "<name of the offending field>",
      "message": "<reason this field failed validation>"
    }
  ]
}

401 Response

{
  "status": "error",
  "code": "UNAUTHORIZED",
  "message": "Authorization header is required, or the token is invalid or expired."
}

403 Response

{
  "status": "error",
  "code": "FORBIDDEN",
  "message": "You do not have permission to perform this action."
}

404 Response

{
  "status": "error",
  "code": "NOT_FOUND",
  "message": "The specified resource does not exist."
}

409 Response

{
  "status": "error",
  "code": "CONFLICT",
  "message": "The request conflicts with the current state of the resource."
}

500 Response

{
  "status": "error",
  "code": "INTERNAL_ERROR",
  "message": "An unexpected error occurred.",
  "trackingId": "4f1c6f2e-8a4b-4c93-b1de-9f2f6f0c2a11"
}

Responses

Status Meaning Description Schema
201 Created LLM proxy created successfully LLMProxy
400 Bad Request Bad Request. Invalid request or validation error. Error
401 Unauthorized Unauthorized. Authentication credentials are missing or invalid. Error
403 Forbidden Forbidden. The authenticated user does not have permission to access this resource. Error
404 Not Found Not Found. The specified resource does not exist. Error
409 Conflict Conflict. The request conflicts with the current state of the resource. Error
500 Internal Server Error Internal Server Error. Error

Response Headers

Status Header Type Format Description
201 Location string uri URL of the newly created resource.

List all LLM proxies

GET /llm-proxies

Code samples

curl -X GET https://localhost:9243/api/v0.9/llm-proxies?projectId=default-project \
  -H 'Authorization: Bearer {access_token}' \
  -H 'Accept: application/json'

Retrieve a list of all LLM proxies for a project. Requires the projectId query parameter.

Authentication

Parameters

Name In Type Required Description
projectId query string true Project ID consisting of the handle (unique slug identifier) of the Project whose resources should be returned.
limit query integer false Maximum number of items to return per page.
offset query integer false Zero-based index of the first item to return.

Detailed descriptions

projectId: Project ID consisting of the handle (unique slug identifier) of the Project whose resources should be returned.

Example responses

200 Response

{
  "count": 2,
  "list": [
    {
      "id": "wso2-con-assistant",
      "displayName": "WSO2 Con Assistant",
      "description": "Customer support assistant",
      "createdBy": "john.doe",
      "context": "/wso2-con-assistant",
      "version": "v1.0",
      "projectId": "default-project",
      "provider": "wso2-openai-provider",
      "status": "deployed",
      "createdAt": "2025-11-25T10:30:00Z",
      "updatedAt": "2025-11-25T10:30:00Z",
      "readOnly": false
    }
  ],
  "pagination": {
    "total": 10,
    "offset": 0,
    "limit": 10
  }
}

401 Response

{
  "status": "error",
  "code": "UNAUTHORIZED",
  "message": "Authorization header is required, or the token is invalid or expired."
}

500 Response

{
  "status": "error",
  "code": "INTERNAL_ERROR",
  "message": "An unexpected error occurred.",
  "trackingId": "4f1c6f2e-8a4b-4c93-b1de-9f2f6f0c2a11"
}

Responses

Status Meaning Description Schema
200 OK List of LLM proxies LLMProxyListResponse
401 Unauthorized Unauthorized. Authentication credentials are missing or invalid. Error
500 Internal Server Error Internal Server Error. Error

Get LLM proxy by unique identifier

GET /llm-proxies/{llmProxyId}

Code samples

curl -X GET https://localhost:9243/api/v0.9/llm-proxies/{llmProxyId} \
  -H 'Authorization: Bearer {access_token}' \
  -H 'Accept: application/json'

Retrieve the complete configuration for a specific LLM proxy.

Authentication

Parameters

Name In Type Required Description
llmProxyId path string true Unique identifier of the LLM proxy

Example responses

200 Response

{
  "id": "wso2-con-assistant",
  "displayName": "WSO2 Con Assistant",
  "description": "Customer support assistant",
  "createdBy": "john.doe",
  "readOnly": false,
  "updatedBy": "john.doe",
  "version": "v1.0",
  "projectId": "default-project",
  "context": "/openai",
  "vhost": "api.openai.com",
  "provider": {
    "id": "wso2-openai-provider",
    "auth": {
      "type": "api-key",
      "header": "X-API-Key"
    }
  },
  "additionalProviders": [
    {
      "id": "anthropic-provider",
      "as": "anthropic-upstream",
      "transformer": {
        "type": "openai-to-anthropic",
        "version": "v1",
        "params": {}
      }
    }
  ],
  "openapi": "openapi: 3.0.3\ninfo:\n  title: Proxy API\n  version: v1.0\npaths: {}\n",
  "globalPolicies": [
    {
      "executionCondition": "request.header.x-custom == 'enabled'",
      "name": "SET_HEADER",
      "params": {
        "key": "MyHeader",
        "value": "MyValue"
      },
      "version": "v1"
    }
  ],
  "operationPolicies": [
    {
      "name": "token-based-ratelimit",
      "version": "v1",
      "executionCondition": "string",
      "paths": [
        {
          "path": "/chat/completions",
          "methods": [
            "GET"
          ],
          "params": {}
        }
      ]
    }
  ],
  "policies": [
    {
      "name": "budgetControl",
      "version": "v1",
      "paths": [
        {
          "path": "/chat/completions",
          "methods": [
            "GET"
          ],
          "params": {}
        }
      ]
    }
  ],
  "security": {
    "enabled": true,
    "apiKey": {
      "enabled": true,
      "key": "X-API-Key",
      "valuePrefix": "Bearer",
      "in": "header"
    }
  },
  "associatedGateways": [
    {
      "id": "prod-eu"
    },
    {
      "id": "prod-us"
    }
  ],
  "createdAt": "2023-10-12T10:30:00Z",
  "updatedAt": "2023-10-12T10:30:00Z"
}

401 Response

{
  "status": "error",
  "code": "UNAUTHORIZED",
  "message": "Authorization header is required, or the token is invalid or expired."
}

404 Response

{
  "status": "error",
  "code": "NOT_FOUND",
  "message": "The specified resource does not exist."
}

500 Response

{
  "status": "error",
  "code": "INTERNAL_ERROR",
  "message": "An unexpected error occurred.",
  "trackingId": "4f1c6f2e-8a4b-4c93-b1de-9f2f6f0c2a11"
}

Responses

Status Meaning Description Schema
200 OK LLM proxy details LLMProxy
401 Unauthorized Unauthorized. Authentication credentials are missing or invalid. Error
404 Not Found Not Found. The specified resource does not exist. Error
500 Internal Server Error Internal Server Error. Error

Update an existing LLM proxy

PUT /llm-proxies/{llmProxyId}

Code samples

curl -X PUT https://localhost:9243/api/v0.9/llm-proxies/{llmProxyId} \
  -H 'Authorization: Bearer {access_token}' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json' \
  -d @payload.json

Update the configuration of an existing LLM proxy.

Payload

{
  "id": "wso2-con-assistant",
  "displayName": "WSO2 Con Assistant",
  "description": "Customer support assistant",
  "version": "v1.0",
  "projectId": "default-project",
  "context": "/openai",
  "vhost": "api.openai.com",
  "provider": {
    "id": "wso2-openai-provider",
    "auth": {
      "type": "api-key",
      "header": "X-API-Key",
      "value": "my-api-key-value"
    }
  },
  "additionalProviders": [
    {
      "id": "anthropic-provider",
      "as": "anthropic-upstream",
      "transformer": {
        "type": "openai-to-anthropic",
        "version": "v1",
        "params": {}
      }
    }
  ],
  "openapi": "openapi: 3.0.3\ninfo:\n  title: Proxy API\n  version: v1.0\npaths: {}\n",
  "globalPolicies": [
    {
      "executionCondition": "request.header.x-custom == 'enabled'",
      "name": "SET_HEADER",
      "params": {
        "key": "MyHeader",
        "value": "MyValue"
      },
      "version": "v1"
    }
  ],
  "operationPolicies": [
    {
      "name": "token-based-ratelimit",
      "version": "v1",
      "executionCondition": "string",
      "paths": [
        {
          "path": "/chat/completions",
          "methods": [
            "GET"
          ],
          "params": {}
        }
      ]
    }
  ],
  "policies": [
    {
      "name": "budgetControl",
      "version": "v1",
      "paths": [
        {
          "path": "/chat/completions",
          "methods": [
            "GET"
          ],
          "params": {}
        }
      ]
    }
  ],
  "security": {
    "enabled": true,
    "apiKey": {
      "enabled": true,
      "key": "X-API-Key",
      "valuePrefix": "Bearer",
      "in": "header"
    }
  },
  "associatedGateways": [
    {
      "id": "prod-eu"
    },
    {
      "id": "prod-us"
    }
  ]
}

Authentication

Parameters

Name In Type Required Description
llmProxyId path string true Unique identifier of the LLM proxy
body body LLMProxy true none

Example responses

200 Response

{
  "id": "wso2-con-assistant",
  "displayName": "WSO2 Con Assistant",
  "description": "Customer support assistant",
  "createdBy": "john.doe",
  "readOnly": false,
  "updatedBy": "john.doe",
  "version": "v1.0",
  "projectId": "default-project",
  "context": "/openai",
  "vhost": "api.openai.com",
  "provider": {
    "id": "wso2-openai-provider",
    "auth": {
      "type": "api-key",
      "header": "X-API-Key"
    }
  },
  "additionalProviders": [
    {
      "id": "anthropic-provider",
      "as": "anthropic-upstream",
      "transformer": {
        "type": "openai-to-anthropic",
        "version": "v1",
        "params": {}
      }
    }
  ],
  "openapi": "openapi: 3.0.3\ninfo:\n  title: Proxy API\n  version: v1.0\npaths: {}\n",
  "globalPolicies": [
    {
      "executionCondition": "request.header.x-custom == 'enabled'",
      "name": "SET_HEADER",
      "params": {
        "key": "MyHeader",
        "value": "MyValue"
      },
      "version": "v1"
    }
  ],
  "operationPolicies": [
    {
      "name": "token-based-ratelimit",
      "version": "v1",
      "executionCondition": "string",
      "paths": [
        {
          "path": "/chat/completions",
          "methods": [
            "GET"
          ],
          "params": {}
        }
      ]
    }
  ],
  "policies": [
    {
      "name": "budgetControl",
      "version": "v1",
      "paths": [
        {
          "path": "/chat/completions",
          "methods": [
            "GET"
          ],
          "params": {}
        }
      ]
    }
  ],
  "security": {
    "enabled": true,
    "apiKey": {
      "enabled": true,
      "key": "X-API-Key",
      "valuePrefix": "Bearer",
      "in": "header"
    }
  },
  "associatedGateways": [
    {
      "id": "prod-eu"
    },
    {
      "id": "prod-us"
    }
  ],
  "createdAt": "2023-10-12T10:30:00Z",
  "updatedAt": "2023-10-12T10:30:00Z"
}

400 Response

{
  "status": "error",
  "code": "VALIDATION_FAILED",
  "message": "The request failed validation.",
  "errors": [
    {
      "field": "<name of the offending field>",
      "message": "<reason this field failed validation>"
    }
  ]
}

401 Response

{
  "status": "error",
  "code": "UNAUTHORIZED",
  "message": "Authorization header is required, or the token is invalid or expired."
}

403 Response

{
  "status": "error",
  "code": "FORBIDDEN",
  "message": "You do not have permission to perform this action."
}

404 Response

{
  "status": "error",
  "code": "NOT_FOUND",
  "message": "The specified resource does not exist."
}

500 Response

{
  "status": "error",
  "code": "INTERNAL_ERROR",
  "message": "An unexpected error occurred.",
  "trackingId": "4f1c6f2e-8a4b-4c93-b1de-9f2f6f0c2a11"
}

Responses

Status Meaning Description Schema
200 OK LLM proxy updated successfully LLMProxy
400 Bad Request Bad Request. Invalid request or validation error. Error
401 Unauthorized Unauthorized. Authentication credentials are missing or invalid. Error
403 Forbidden Forbidden. The authenticated user does not have permission to access this resource. Error
404 Not Found Not Found. The specified resource does not exist. Error
500 Internal Server Error Internal Server Error. Error

Delete an LLM proxy

DELETE /llm-proxies/{llmProxyId}

Code samples

curl -X DELETE https://localhost:9243/api/v0.9/llm-proxies/{llmProxyId} \
  -H 'Authorization: Bearer {access_token}' \
  -H 'Accept: application/json'

Remove an LLM proxy.

Authentication

Parameters

Name In Type Required Description
llmProxyId path string true Unique identifier of the LLM proxy

Example responses

400 Response

{
  "status": "error",
  "code": "VALIDATION_FAILED",
  "message": "The request failed validation.",
  "errors": [
    {
      "field": "<name of the offending field>",
      "message": "<reason this field failed validation>"
    }
  ]
}

401 Response

{
  "status": "error",
  "code": "UNAUTHORIZED",
  "message": "Authorization header is required, or the token is invalid or expired."
}

403 Response

{
  "status": "error",
  "code": "FORBIDDEN",
  "message": "You do not have permission to perform this action."
}

404 Response

{
  "status": "error",
  "code": "NOT_FOUND",
  "message": "The specified resource does not exist."
}

500 Response

{
  "status": "error",
  "code": "INTERNAL_ERROR",
  "message": "An unexpected error occurred.",
  "trackingId": "4f1c6f2e-8a4b-4c93-b1de-9f2f6f0c2a11"
}

Responses

Status Meaning Description Schema
204 No Content LLM proxy deleted successfully None
400 Bad Request Bad Request. Invalid request or validation error. Error
401 Unauthorized Unauthorized. Authentication credentials are missing or invalid. Error
403 Forbidden Forbidden. The authenticated user does not have permission to access this resource. Error
404 Not Found Not Found. The specified resource does not exist. Error
500 Internal Server Error Internal Server Error. Error

Create a new API key for an LLM proxy

POST /llm-proxies/{llmProxyId}/api-keys

Code samples

curl -X POST https://localhost:9243/api/v0.9/llm-proxies/{llmProxyId}/api-keys \
  -H 'Authorization: Bearer {access_token}' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json' \
  -d @payload.json

Generates a new API key for the specified LLM proxy. The generated key is broadcasted to all gateways in the organization and can be used to authenticate requests to the LLM proxy when API key validation is enabled.

Payload

{
  "id": "production-key",
  "displayName": "Production Key",
  "expiresAt": "2026-12-31T23:59:59Z",
  "issuer": "api-platform-devportal",
  "allowedTargets": "dev_gateway,test_gateway"
}

Authentication

Parameters

Name In Type Required Description
llmProxyId path string true Unique identifier of the LLM proxy
body body CreateLLMProxyAPIKeyRequest true API key creation details

Example responses

201 Response

{
  "status": "success",
  "message": "API key created and broadcasted to gateways successfully",
  "id": "production-key",
  "apiKey": "REDACTED_API_KEY"
}

400 Response

{
  "status": "error",
  "code": "VALIDATION_FAILED",
  "message": "The request failed validation.",
  "errors": [
    {
      "field": "<name of the offending field>",
      "message": "<reason this field failed validation>"
    }
  ]
}

401 Response

{
  "status": "error",
  "code": "UNAUTHORIZED",
  "message": "Authorization header is required, or the token is invalid or expired."
}

404 Response

{
  "status": "error",
  "code": "NOT_FOUND",
  "message": "The specified resource does not exist."
}

500 Response

{
  "status": "error",
  "code": "INTERNAL_ERROR",
  "message": "An unexpected error occurred.",
  "trackingId": "4f1c6f2e-8a4b-4c93-b1de-9f2f6f0c2a11"
}

503 Response

{
  "status": "error",
  "code": "GATEWAY_CONNECTION_UNAVAILABLE",
  "message": "No gateway connections are currently available.",
  "trackingId": "4f1c6f2e-8a4b-4c93-b1de-9f2f6f0c2a11"
}

Responses

Status Meaning Description Schema
201 Created API key created successfully CreateLLMProxyAPIKeyResponse
400 Bad Request Bad Request. Invalid request or validation error. Error
401 Unauthorized Unauthorized. Authentication credentials are missing or invalid. Error
404 Not Found Not Found. The specified resource does not exist. Error
500 Internal Server Error Internal Server Error. Error
503 Service Unavailable Service Unavailable. No gateway connections are currently available to service this request. Error

Response Headers

Status Header Type Format Description
201 Location string uri URL of the newly created resource.

List API keys for an LLM proxy

GET /llm-proxies/{llmProxyId}/api-keys

Code samples

curl -X GET https://localhost:9243/api/v0.9/llm-proxies/{llmProxyId}/api-keys \
  -H 'Authorization: Bearer {access_token}' \
  -H 'Accept: application/json'

Returns all API keys associated with the specified LLM proxy. The plain key value is never returned.

Authentication

Parameters

Name In Type Required Description
llmProxyId path string true Unique identifier of the LLM proxy
limit query integer false Maximum number of items to return per page.
offset query integer false Zero-based index of the first item to return.

Example responses

200 Response

{
  "list": [
    {
      "id": "string",
      "displayName": "string",
      "maskedApiKey": "string",
      "status": "active",
      "createdAt": "2019-08-24T14:15:22Z",
      "createdBy": "john.doe",
      "updatedAt": "2019-08-24T14:15:22Z",
      "expiresAt": "2019-08-24T14:15:22Z",
      "issuer": "api-platform-devportal",
      "allowedTargets": "string"
    }
  ],
  "count": 0,
  "pagination": {
    "total": 10,
    "offset": 0,
    "limit": 10
  }
}

401 Response

{
  "status": "error",
  "code": "UNAUTHORIZED",
  "message": "Authorization header is required, or the token is invalid or expired."
}

404 Response

{
  "status": "error",
  "code": "NOT_FOUND",
  "message": "The specified resource does not exist."
}

500 Response

{
  "status": "error",
  "code": "INTERNAL_ERROR",
  "message": "An unexpected error occurred.",
  "trackingId": "4f1c6f2e-8a4b-4c93-b1de-9f2f6f0c2a11"
}

Responses

Status Meaning Description Schema
200 OK List of API keys retrieved successfully LLMProxyAPIKeyListResponse
401 Unauthorized Unauthorized. Authentication credentials are missing or invalid. Error
404 Not Found Not Found. The specified resource does not exist. Error
500 Internal Server Error Internal Server Error. Error

Delete an API key for an LLM proxy

DELETE /llm-proxies/{llmProxyId}/api-keys/{apiKeyId}

Code samples

curl -X DELETE https://localhost:9243/api/v0.9/llm-proxies/{llmProxyId}/api-keys/{apiKeyId} \
  -H 'Authorization: Bearer {access_token}' \
  -H 'Accept: application/json'

Deletes the key from the database and broadcasts a revoke event to the allowed gateways.

Authentication

Parameters

Name In Type Required Description
llmProxyId path string true Unique identifier of the LLM proxy
apiKeyId path string true Name of the API key to delete

Example responses

401 Response

{
  "status": "error",
  "code": "UNAUTHORIZED",
  "message": "Authorization header is required, or the token is invalid or expired."
}

404 Response

{
  "status": "error",
  "code": "NOT_FOUND",
  "message": "The specified resource does not exist."
}

500 Response

{
  "status": "error",
  "code": "INTERNAL_ERROR",
  "message": "An unexpected error occurred.",
  "trackingId": "4f1c6f2e-8a4b-4c93-b1de-9f2f6f0c2a11"
}

503 Response

{
  "status": "error",
  "code": "GATEWAY_CONNECTION_UNAVAILABLE",
  "message": "No gateway connections are currently available.",
  "trackingId": "4f1c6f2e-8a4b-4c93-b1de-9f2f6f0c2a11"
}

Responses

Status Meaning Description Schema
204 No Content API key deleted successfully None
401 Unauthorized Unauthorized. Authentication credentials are missing or invalid. Error
404 Not Found Not Found. The specified resource does not exist. Error
500 Internal Server Error Internal Server Error. Error
503 Service Unavailable Service Unavailable. No gateway connections are currently available to service this request. Error