AI Workspace is Now Self-Managed: Run Your AI Control Plane On Your Own Infrastructure

Running AI traffic through your API gateway makes sense at first. Many API management platforms implemented this pattern when AI traffic first came into scope, so it wasn’t unheard of. It also seemed to make a lot of sense since a large language model (LLM) endpoint is an HTTP endpoint, you already run a gateway, and you already have somewhere to attach policies. As AI gateways have evolved into a separate entity, and the further we got into building AI Workspace, the clearer it became that the control plane for AI traffic doesn't belong inside the control plane for APIs.

First, what these two things are

AI adoption has outrun AI governance in most enterprises. Teams connect LLMs, agents and Model Context Protocol (MCP) servers faster than platform teams can secure them, and nobody has a single list of what's connected. WSO2 API Platform has two components for that problem: the AI Gateway and AI Workspace.

The AI Gateway is the runtime. It sits in the request path, routes traffic to model providers, and enforces whatever controls you've configured. The AI Workspace is the control plane: a dedicated place to govern every service your AI applications touch, from models and gateways through to MCP interactions. Think of the workspace as where the rules are written and the gateway as where they're applied (in the exact same way that an API management control planes works with API data planes/gateways).

As of this release, AI Workspace is available as a 100% self-managed deployment. Everything you configure in the Cloud version now runs on infrastructure you own, including air-gapped networks. We also have three new features that ship with it, including artifacts created directly on a gateway now sync up into the workspace, you can author your own LLM provider templates, and policies can be attached at global level on a provider or proxy.

Why we didn't put AI governance in the API control plane

One control plane is easier to build and easier to sell, so we spent real time on that option before deciding to go down the path of separation. Four things pushed us the other way, and they aren't equally important.

The lifecycles don't line up. An API changes when its contract changes: a new version, a deprecated field, consumers to migrate. AI traffic changes for reasons that have nothing to do with contracts. Someone swaps a model for a cheaper one, tightens a system prompt, or raises a token ceiling because a team's agent got popular. None of that needs a consumer migration, and most of it happens weekly rather than quarterly. Push it through an API lifecycle and you get ceremony without safety.

Different people own them. The team deciding which models are approved and what they cost is usually not the team publishing APIs. Roles inside one portal solve part of that, but not the vocabulary problem: the AI side reasons about models, prompts, tokens and guardrails, while the API side reasons about resources, versions, subscriptions and consumers. If each of these roles share a navigation tree, you end up with each group working around the other's concepts to reach their own which isn’t ideal for either user group.

The reports are different. An API dashboard answers throughput, latency and which consumer is hammering you; an AI dashboard has to answer cost first. That's why rate limiting here includes token-based and LLM cost-based limits alongside the request-count ones, because a request count tells you nothing about a bill when one call might be a two-token ping or a 100,000-token document summary.

Guardrails carry more weight than API policies. Guardrails that implement personally identifiable information (PII) masking, prompt injection detection, content safety scoring, JSON schema validation on responses, and semantic tool filtering on MCP traffic are very specific to AI use cases. This type of policy enforcement decides whether AI traffic is safe to serve, and almost none of it maps onto a conventional API gateway or the policies within them. To enforce, they need request and response bodies parsed, model-aware context to judge them against, and somewhere to fail closed on content rather than on a counter. An API policy chain authenticates, transforms and throttles, then hands the payload onward rather than judging it. Skim the full policy set and you'll see how little of it has an API-management equivalent.

Of course, two control planes has a cost: two places to look, two authentication surfaces, and real work correlating an incident that crosses both. We'd still rather pay it than govern an AI estate through a policy model built for API contracts.

Why running the control plane yourself matters

Sovereignty requirements are turning up in most regulated conversations we're having, from banks preparing for the Digital Operational Resilience Act (DORA) to governments writing open-source-first procurement rules. The EU AI Act, NIS2 and eIDAS 2.0 all point the same way: regulated buyers increasingly have to prove control over how AI decisions get made, not just where data is stored.

Even if gateways are hosted on sovereign infrastructure, a vendor-hosted control plane is still a problem if you require everything to be self-hosted for regulatory or organizational-level reasons. Your gateway could sit in your data center while your policy definitions, routing decisions and audit trail lived in ours. For a bank or a government agency, a local enforcement point isn't enough if the record of what was enforced, and who changed it, sits outside their boundary. Now none of it has to leave.

What you actually run

The platform has three deployment options: we run everything in Cloud, we run the control plane and you run the gateways in Hybrid, and in Self-Managed you run the lot. AI Workspace has only ever existed in the first two and now users have the freedom to use all three that the rest of WSO2 API Platform supports.

Here's what you're choosing between now if you run self-managed: The Docker Compose stack runs on a single Linux host, and it's the shape the docs point at air-gapped or tightly controlled networks. The alternative is the ai-workspace Helm chart on Kubernetes, for teams who already run a cluster and want rolling upgrades. Authentication points at whatever you already use, including Asgardeo, Microsoft Entra ID, or any OIDC-compliant identity provider. The deployment guide covers versions, database setup and hardening.

Artifacts you create on the gateway now show up in the workspace

Until now the flow ran one way. You modeled an artifact in AI Workspace and deployed it down to a gateway, which works well for platform teams and badly for anyone who lives in YAML and command-line interfaces. Gateways can also run standalone, configured directly through the gateway's management API, and plenty of teams start there because it's the fastest path to serving traffic. So what was the cost? Invisibility. A proxy created that way served requests happily but never appeared in the workspace, so the people accountable for AI governance had no idea it existed.

That flow now runs in both directions. Create an LLM Provider Template, LLM Provider, LLM Proxy or MCP Proxy on the gateway and it takes effect immediately, then syncs up into AI Workspace as a read-only copy. One gateway setting controls it, deployment_sync_enabled, and it's on by default. We’ve pushed this into the self-managed product first, so if you're running AI Workspace in Cloud, the reverse flow will be there in coming cloud releases.

Read-only is the design decision that keeps this from becoming a mess of concurrency and duplicate writes. In the workspace you can edit what doesn't affect how the gateway runs the artifact, like display name, description and OpenAPI definitions. You can't change what it uses to serve traffic, meaning upstreams, auth, routing and policies. Whatever created the artifact keeps governing it, and the other side gets visibility. The sync documentation covers the API calls and what happens when the workspace is unreachable, which for an air-gapped deployment is the section to read first.

Custom LLM provider templates

We ship templates for seven services: OpenAI, Azure OpenAI, Azure AI Foundry, Anthropic, Google Gemini, Mistral AI and AWS Bedrock. That list will always trail the market, and it will never include the model your research team is self-hosting behind an internal endpoint.

So templates are now something you can author. A template describes how an upstream service behaves, separately from the connection to it. Create one from scratch and it starts at v1.0, or base it on a built-in template as a new version. If you're self-hosting inference or running a regional provider we don't cover, this is the path that doesn't require waiting on us. The template configuration guide has the details.

Global policy scope on providers and proxies

Attaching a guardrail endpoint by endpoint is fine until you have thirty endpoints and a compliance requirement covering all of them. Policies can now be attached at global level on a provider or proxy, meaning every endpoint of that resource, and rate limits are covered by the same model rather than just guardrails.

Here's the part worth internalizing. A global rate limit keeps one bucket for the whole resource, so a limit of 100 requests an hour is exhausted by 60 calls to one endpoint plus 40 to another. Global policies also evaluate first, which makes that limit a hard ceiling: it counts every attempt, including ones a tighter per-resource limit goes on to reject. Provider policies then act as a baseline for every proxy using them, so an application team can add controls without shedding the ones you set upstream.

Getting it running

Self-managed AI Workspace 1.0.0 is available now. Ready to run it yourself? Start with the deployment shapes overview to pick between Docker Compose and Helm, and read About this release for the full feature list, including secrets management and Git-based delivery through the ap command-line interface (our API Platform CLI package).

And if you're already running the AI Gateway standalone, the shortest path to seeing what changed is to turn on deployment_sync_enabled, point the gateway at a workspace, and watch what you've already built appear.