Skip to content
Edit page
Copy page

LLM header routing

Use llm-header-router when the application, or an earlier policy in the chain, must name the provider explicitly rather than leave the choice to the gateway. The router reads a request header, matches its value against an ordered mapping, and publishes the selected provider name.

Unlike the round robin policies, the header router selects a single provider rather than distributing across a pool, so it carries no suspension or failover behavior.

Parameters

Parameter Required Default Description
mappings Yes None Ordered list of header values and effective provider names. At least one mapping is required.
headerName No x-provider Header used for provider selection. Header-name lookup is case-insensitive.
defaultProvider No Unset Provider selected when the header is missing, empty, or unmatched. If unset, the primary provider is used.

The router has the following selection behavior:

  • Uses only the first value when the header appears more than once
  • Trims leading and trailing whitespace from the value
  • Matches configured values case-insensitively
  • Rejects duplicate mapping values case-insensitively
  • Preserves a non-empty provider selection made by an earlier policy
  • Leaves the routing header on the upstream request

The header router does two things. It sets the selected provider as the upstream, so the request reaches that provider whether or not a transformer is attached. It also records which provider was selected, so only that provider's credential and transformer policies are applied.

Policy reference

This policy is documented in the Policy Hub, the versioned reference for every API Platform policy. See LLM Header Router for its complete configuration.

  • Multi-provider routing — the worked configuration this policy appears in, including when a provider needs a transformer and when it does not.
  • Load balancing and failover — the round robin policies, for distributing across a pool instead of naming one provider.
  • Multi model routing — the use-case page for distributing traffic across models.