Copy page

Version: STS 2026.11.0

Upgrade from v1.0.0 to v2026.11.0

This guide upgrades an existing WSO2 Agent Manager v1.0.0 Helm installation to the short-term support (STS) release 2026.11.0, or to one of its updates. The commands use the quick-start release names and namespaces. Substitute the names from your installation, and repeat the gateway steps for every gateway in every organization and environment, including the <org>-<env>-egress gateway of an environment with a separate egress gateway.

Release 2026.11.0 is available only from registry.wso2.com, so every chart and image in this guide comes from the registry, with your registry credentials, except the Gateway Operator chart, which still comes from ghcr.io.

Breaking changes

Change	Impact	Required action
OpenChoreo moves from 1.2.0 to 1.2.8	The CRDs are unchanged. A Helm upgrade of the Control Plane and Observability Plane reverts the service-account `client_id` entitlement patch.	Upgrade the four planes and re-apply the patch, as described in Upgrade platform dependencies.
ThunderID moves from 1.0.0 to 1.0.2	Platform Thunder and each environment Thunder are separate releases.	Upgrade each instance separately.
kgateway moves from v2.2.1 to v2.3.11	The `kgateway-crds` chart updates the kgateway CRDs.	Upgrade `kgateway-crds`, then `kgateway`.
The Observability Plane enables an ingress NetworkPolicy	Monitor runs and run logs fail with `Connection refused` unless the Agent Manager observer is exempt. Log collection moves to a TLS passthrough route.	Upgrade the Observability Plane with the values in Upgrade platform dependencies.
The observability modules move to logs 0.5.5, tracing 0.8.1, and metrics 0.8.1	OpenSearch moves from 3.3.0 to 3.9.0 and cannot be rolled back without a snapshot. The Prometheus Operator CRDs change, and Prometheus and Alertmanager move to PersistentVolumeClaims, so earlier metrics are not kept.	Take an OpenSearch snapshot, confirm a default StorageClass, and apply the Prometheus Operator CRDs before the upgrade, as described in Upgrade platform dependencies.
Gateway Operator moves from chart 0.11.0 to 0.11.3	Only the `llmprovidertemplates` CRD changes. Upgrading the operator does not redeploy existing gateways.	Apply the CRDs, upgrade the operator, and then trigger each gateway explicitly (Step 7).
API Platform gateway moves from images 1.2.1 to 2026.10.0	The gateway controller's SQLite schema moves from version 4 to version 6, and the new controller refuses a version 4 database. Agent Manager does not resend gateway configuration to an empty controller.	Upgrade each controller database to version 5 in place before its gateway redeploys (Step 7). Gateway 2026.10.0 then migrates it to version 6 on its first start. Do not delete it.
Gateway 2026.10.0 changes three defaults	Gateways present a default client certificate to backends, return `x-request-id`, and serve `/.well-known/jwks` on every host.	To opt out, set `present_default_identity`, `return_request_id_in_response`, or `jwks_endpoint_enabled` to `false`.
Charts and images move from `ghcr.io` to `registry.wso2.com`	Charts, Agent Manager images, and the gateway chart and images need registry credentials and pull Secrets.	Complete Step 1, including the pull Secrets and `AMP_IMAGE_ARGS`.
New LLM proxies take their API key header from the LLM Service Provider	New proxies expect the provider's header, such as `X-API-Key`. Existing proxies keep `API-Key`. Agents that send `API-Key` to a new proxy get `401`.	Send the header shown in the agent's LLM configuration in the Console.
The Agent Manager database moves from migration 0043 to 0050	Migrations 0044 to 0050 run in one transaction. v1.0.0 refuses to start against a database that contains them.	Back up PostgreSQL first. A chart rollback alone cannot restore v1.0.0.
The gateway catch-all route request timeout moves from 30 seconds to unlimited	Long-running and streaming requests are no longer cut off at 30 seconds. The gateway's 5-minute idle timeout still applies.	Review any client behavior that relied on the 30-second limit.
The platform Thunder bootstrap runs only at install time	The target adds a refresh-token lifetime and login-screen branding to the Console, CLI, and MCP clients, and renames the Agent Manager MCP resource server to `amp-agent-manager-mcp-resource-srv`. An upgrade does not apply these.	Existing clients and resource servers keep their v1.0.0 settings and names.

Prerequisites

Helm 3.14 or later, for --reset-then-reuse-values. The target installer uses Helm 3.19.4.
kubectl access to every cluster that runs an Agent Manager component.
Your registry credentials for registry.wso2.com.
Every cluster node can pull from registry.wso2.com. Pull Secrets cover the namespaces this guide names, but not the namespaces OpenChoreo creates for agent workloads. Those pulls use the node's container runtime credentials. On k3d, the installer writes them to /etc/rancher/k3s/registries.yaml only when it creates the cluster, so a cluster created for the community release does not have them.
sqlite3 on the machine where you run the commands.
A maintenance window. Each gateway is unavailable while its controller is stopped and redeployed.

Step 1: Set variables and record the installation

Set the release variables. The rest of the guide uses only these variables and the gateway variables that you set after you sign in.

export PROJECT="<project>"
export VERSION="2026.11.0"
export REG="oci://registry.wso2.com/$PROJECT/sts/helm-charts"
export GW_REGISTRY="registry.wso2.com/$PROJECT/sts"
GW_REGISTRY_ARGS=(
  --set-string gateway.helm.chartName="oci://$GW_REGISTRY/helm-charts/gateway"
  --set-string gateway.helm.registryCredentialsSecret.name=wso2-gateway-registry-credentials
  --set-string "gateway.values.imagePullSecrets[0]=wso2-gateway-registry-credentials"
  --set-string gateway.values.gateway.controller.image.repository="$GW_REGISTRY/gateway-controller"
  --set-string gateway.values.gateway.gatewayRuntime.image.repository="$GW_REGISTRY/gateway-runtime"
)
AMP_IMAGE_ARGS=(--set-string global.ampImageRegistry="registry.wso2.com/$PROJECT/sts")

Replace <project> with wso2-agent-manager for a subscription, or wso2-agent-manager-evaluations for an evaluation. To install an update instead of the release, set VERSION to it, for example 2026.11.0-u2. See List the Published Versions. The gateway comes from the same registry and uses the same credentials.

Then set the shared variables and record the installation:

export AMP_NS="wso2-amp"
export DP_NS="openchoreo-data-plane"
export GATEWAY_CONTEXT="<gateway-cluster-context>"

helm version
helm list -A
kubectl --context "$GATEWAY_CONTEXT" get apigateways -A

Replace <gateway-cluster-context> with the Kubernetes context of the cluster that runs the Gateway Operator and your gateways. On a single-cluster installation, this is the output of kubectl config current-context. For development testing, set VERSION to the exact snapshot under test.

Record each Helm release, namespace, chart version, and Kubernetes context. Record every APIGateway with its namespace. The quick-start installer names them api-platform-<org>-<env> in the <org>-<env> namespace, and an egress gateway api-platform-<org>-<env>-egress in the <org>-<env>-egress namespace.

Read your registry credentials without storing them in the shell history, and sign Helm in to the registry. The commands work in both bash and zsh. Quote the username wherever you use it, because registry usernames can contain $:

printf 'Registry username: '; read -r AMP_REGISTRY_USERNAME
printf 'Registry token: '; read -rs AMP_REGISTRY_PASSWORD; echo
printf '%s' "$AMP_REGISTRY_PASSWORD" | helm registry login registry.wso2.com \
  --username "$AMP_REGISTRY_USERNAME" --password-stdin
helm show chart "$REG/wso2-agent-manager" --version "$VERSION"
helm show values "$REG/wso2-amp-api-platform-gateway-extension" --version "$VERSION" \
  | grep -A3 '^gatewayRelease:'

The second command prints the operator, gateway chart, and gateway image versions this release pins. Set the variables to them; chart and image versions can differ in an update. For release 2026.11.0:

export GW_OPERATOR_VERSION="0.11.3"
export GW_VERSION="2026.10.0"
export GW_IMAGE_VERSION="2026.10.0"
helm show chart "oci://$GW_REGISTRY/helm-charts/gateway" --version "$GW_VERSION"

The last command prints the gateway chart. Write the credentials to files that only you can read, so the token never appears on a command line:

SECRET_DIR="$(mktemp -d)"
chmod 700 "$SECRET_DIR"
printf '{"auths":{"registry.wso2.com":{"auth":"%s"}}}' \
  "$(printf '%s:%s' "$AMP_REGISTRY_USERNAME" "$AMP_REGISTRY_PASSWORD" | base64 | tr -d '\n')" \
  > "$SECRET_DIR/config.json"
printf '%s' "$AMP_REGISTRY_USERNAME" > "$SECRET_DIR/username"
printf '%s' "$AMP_REGISTRY_PASSWORD" > "$SECRET_DIR/password"

Create the wso2-registry-credentials Secret in the namespaces that run Agent Manager images, including the workflows-<org> namespace of every organization. In a multi-cluster installation, run the loop against each cluster that has these namespaces, with kubectl --context:

for ns in "$AMP_NS" openchoreo-observability-plane \
    $(kubectl get namespaces -o name | sed -n 's|^namespace/\(workflows-.*\)|\1|p'); do
  kubectl create secret generic wso2-registry-credentials \
    --namespace "$ns" \
    --type=kubernetes.io/dockerconfigjson \
    --from-file=.dockerconfigjson="$SECRET_DIR/config.json" \
    --dry-run=client -o yaml | kubectl apply -f -
done

The gateway has its own Secret, wso2-gateway-registry-credentials. The Gateway Operator reads it from its namespace to pull the gateway chart, and each gateway reads it from its own namespace to pull its images. Besides the Docker configuration, the Secret holds plain username and password keys, which the operator uses for the chart. Create it in the operator namespace and in every namespace that contains an APIGateway, egress gateways included, then delete the files:

for ns in "$DP_NS" $(kubectl --context "$GATEWAY_CONTEXT" get apigateways -A \
    -o jsonpath='{range .items[*]}{.metadata.namespace}{"\n"}{end}' | sort -u); do
  kubectl --context "$GATEWAY_CONTEXT" create secret generic wso2-gateway-registry-credentials \
    --namespace "$ns" \
    --type=kubernetes.io/dockerconfigjson \
    --from-file=.dockerconfigjson="$SECRET_DIR/config.json" \
    --from-file=username="$SECRET_DIR/username" \
    --from-file=password="$SECRET_DIR/password" \
    --dry-run=client -o yaml | kubectl --context "$GATEWAY_CONTEXT" apply -f -
done
rm -rf "$SECRET_DIR"

Step 2: Back up

Pause deployments, promotions, and configuration changes for the maintenance window.

Save each release's history and user-supplied values. These files can contain credentials, so keep them out of source control:

umask 077
mkdir -p upgrade-backup
helm history amp -n "$AMP_NS" > upgrade-backup/amp-history.txt
helm get values amp -n "$AMP_NS" -o yaml > upgrade-backup/amp-user-values.yaml

Repeat for the other releases recorded in Step 1.

Back up the Agent Manager database. This dump is the only way back to v1.0.0. The quick-start PostgreSQL instance requires a password, which is stored in the amp-postgresql Secret:

export PGPW="$(kubectl get secret amp-postgresql -n "$AMP_NS" -o jsonpath='{.data.password}' | base64 -d)"
kubectl exec -n "$AMP_NS" amp-postgresql-0 -- env PGPASSWORD="$PGPW" \
  pg_dump -U agentmanager -d agentmanager -Fc \
  > upgrade-backup/ams-1.0.0.dump

Check that the dump file is not empty. For an external database, use its own backup tooling. Test the restore in a separate database.

For each gateway, back up the controller data and its encryption key Secret. The key is mounted from the gateway-encryption-keys Secret, not stored on the volume. This backup is the recovery point for the 1.2.1 controller. The loop covers every namespace that contains an APIGateway, egress gateways included:

for GW_NS in $(kubectl --context "$GATEWAY_CONTEXT" get apigateways -A \
    -o jsonpath='{range .items[*]}{.metadata.namespace}{"\n"}{end}' | sort -u); do
  GWC=$(kubectl --context "$GATEWAY_CONTEXT" get deploy -n "$GW_NS" -o name | grep gateway-controller)
  kubectl --context "$GATEWAY_CONTEXT" exec -n "$GW_NS" "$GWC" -- tar cf - -C /app data \
    > "upgrade-backup/gw-data-$GW_NS.tar"
  kubectl --context "$GATEWAY_CONTEXT" get secret gateway-encryption-keys -n "$GW_NS" -o yaml \
    > "upgrade-backup/gw-encryption-keys-$GW_NS.yaml"
done
ls -l upgrade-backup/gw-*

Each gw-data-*.tar file must be larger than zero bytes.

Record a baseline: invoke representative agents, and note existing traces, monitor results, and API keys in use.

Step 3: Upgrade OpenChoreo, the observability modules, and Thunder

Follow Upgrade platform dependencies through its environment Thunder step. Leave the Gateway Operator at 0.11.0 until Step 6.

Step 4: Upgrade platform resources, then Agent Manager

Upgrade platform resources first. The target Agent Manager attaches a new a2a-gateway-route trait to A2A agents, and only the target platform resources chart defines it.

AMP_IMAGE_ARGS points global.ampImageRegistry at the sts path. Without it, --reset-then-reuse-values keeps the v1.0.0 value and the pods fail with ImagePullBackOff. Step 5 uses it too:

helm upgrade amp-platform-resources \
  "$REG/wso2-amp-platform-resources-extension" \
  --version "$VERSION" -n default \
  --reset-then-reuse-values --timeout 15m --wait

helm upgrade amp "$REG/wso2-agent-manager" \
  --version "$VERSION" -n "$AMP_NS" \
  --reset-then-reuse-values "${AMP_IMAGE_ARGS[@]}" \
  --timeout 15m --wait

The amp-db-migration post-upgrade hook applies migrations 0044 to 0050. Kubernetes deletes the job 300 seconds after it finishes, so read its logs right away:

kubectl logs -n "$AMP_NS" job/amp-db-migration
kubectl exec -n "$AMP_NS" amp-postgresql-0 -- env PGPASSWORD="$PGPW" \
  psql -U agentmanager -d agentmanager -tc \
  "select max(id) from migration_history"

The query returns 0050. If the job fails, the transaction leaves the database at 0043. Fix the cause, usually database connectivity or credentials, and rerun the same helm upgrade amp command.

Step 5: Upgrade the observability and evaluation extensions

Skip any extension that is not installed.

helm upgrade amp-observability-traces "$REG/wso2-amp-observability-extension" \
  --version "$VERSION" -n openchoreo-observability-plane \
  --reset-then-reuse-values "${AMP_IMAGE_ARGS[@]}" \
  --timeout 15m --wait

helm upgrade amp-evaluation-extension "$REG/wso2-amp-evaluation-extension" \
  --version "$VERSION" -n openchoreo-workflow-plane \
  --reset-then-reuse-values "${AMP_IMAGE_ARGS[@]}" \
  --timeout 15m --wait

Step 6: Upgrade the gateway extensions and the Gateway Operator

Upgrade each gateway extension release, including the api-platform-<org>-<env>-egress release of an egress gateway. This does not redeploy the gateway. Its pre-upgrade bootstrap job calls Agent Manager, which must be running the target version.

Each gateway extension release has the same name and namespace as its APIGateway, so the loop upgrades one release per APIGateway:

for gw in $(kubectl --context "$GATEWAY_CONTEXT" get apigateways -A \
    -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}{"\n"}{end}'); do
  helm upgrade "${gw#*/}" \
    "$REG/wso2-amp-api-platform-gateway-extension" \
    --version "$VERSION" -n "${gw%/*}" --kube-context "$GATEWAY_CONTEXT" \
    --reset-then-reuse-values --timeout 15m --wait
done

Apply the operator CRDs. Helm does not upgrade CRDs in a chart's crds/ directory. The new CRDs keep serving the existing versions.

helm show crds \
  oci://ghcr.io/wso2/api-platform/helm-charts/gateway-operator \
  --version "$GW_OPERATOR_VERSION" \
  | kubectl --context "$GATEWAY_CONTEXT" apply --server-side --force-conflicts -f -

Upgrade the operator and pin the gateway chart and both gateway images. Setting only the chart version leaves the old images in place. GW_REGISTRY_ARGS also points the operator at the registry, its credentials, and the gateway image repositories, which the existing release sets to ghcr.io.

helm upgrade gateway-operator \
  oci://ghcr.io/wso2/api-platform/helm-charts/gateway-operator \
  --version "$GW_OPERATOR_VERSION" -n "$DP_NS" \
  --kube-context "$GATEWAY_CONTEXT" --reset-then-reuse-values \
  "${GW_REGISTRY_ARGS[@]}" \
  --set-string gateway.helm.chartVersion="$GW_VERSION" \
  --set-string gateway.values.gateway.controller.image.tag="$GW_IMAGE_VERSION" \
  --set-string gateway.values.gateway.gatewayRuntime.image.tag="$GW_IMAGE_VERSION" \
  --timeout 15m --wait

Chart 0.11.3 runs operator image 0.11.1. Existing gateways keep running 1.2.1 after this step.

Step 7: Upgrade each controller database in place and redeploy its gateway

The target controller refuses a version 4 database, and Agent Manager does not resend routes, LLM providers, LLM proxies, or MCP proxies to an empty controller. Upgrade each existing database in place instead. The only schema change between version 4 and version 5 is a new agents table, so the upgrade adds that table and sets the schema version. Gateway 2026.10.0 uses schema version 6, and migrates a version 5 database to version 6 on its first start, but it refuses version 4. Run this step for one gateway at a time, egress gateways included. You need sqlite3 on the machine where you run the commands.

danger

Confirm the Step 2 backups and the Step 4 migration before you continue. Keep the gateway-encryption-keys Secret. Without it, the controller cannot decrypt the stored secrets.

Set the gateway to work on:

export GW_NS="default-default"
export GW_NAME="api-platform-default-default"
export GW_DIR="upgrade-backup/gw-$GW_NS"
mkdir -p "$GW_DIR"
GWC=$(kubectl --context "$GATEWAY_CONTEXT" get deploy -n "$GW_NS" -o name | grep gateway-controller)

Stop the controller process while keeping its pod and volume. The override replaces the controller command with sleep, so nothing writes to the database. First confirm that the container does not already set a command, because removing the override later deletes the field. The first command prints nothing:

kubectl --context "$GATEWAY_CONTEXT" get -n "$GW_NS" "$GWC" \
  -o jsonpath='{.spec.template.spec.containers[0].command}'
kubectl --context "$GATEWAY_CONTEXT" patch -n "$GW_NS" "$GWC" --type=json -p \
  '[{"op":"add","path":"/spec/template/spec/containers/0/command","value":["sh","-c","sleep 86400"]}]'
kubectl --context "$GATEWAY_CONTEXT" get pods -n "$GW_NS" -w | grep gateway-controller

Stop watching once a single controller pod is Running. It stays 0/1 ready because the controller is not running. Copy the database out of that pod:

POD=$(kubectl --context "$GATEWAY_CONTEXT" get pods -n "$GW_NS" --no-headers \
  | awk '/gateway-controller/ && $3=="Running"{print $1}')
kubectl --context "$GATEWAY_CONTEXT" exec -n "$GW_NS" "$POD" -c gateway-controller -- ps
kubectl --context "$GATEWAY_CONTEXT" exec -n "$GW_NS" "$POD" -c gateway-controller -- \
  sh -c 'cd /app/data && tar cf - gateway.db*' > "$GW_DIR/gateway-v4.tar"

The ps output lists sleep and no gateway-controller process. Upgrade the copy to version 5:

tar xf "$GW_DIR/gateway-v4.tar" -C "$GW_DIR"
sqlite3 "$GW_DIR/gateway.db" 'PRAGMA wal_checkpoint(TRUNCATE);'
sqlite3 "$GW_DIR/gateway.db" <<'SQL'
CREATE TABLE IF NOT EXISTS agents (
    uuid TEXT NOT NULL,
    gateway_id TEXT NOT NULL,
    configuration TEXT NOT NULL,
    signed_public_card TEXT,
    signed_protected_card TEXT,
    PRIMARY KEY (gateway_id, uuid),
    FOREIGN KEY(gateway_id, uuid) REFERENCES artifacts(gateway_id, uuid) ON DELETE CASCADE
);
PRAGMA user_version = 5;
SQL
sqlite3 "$GW_DIR/gateway.db" 'PRAGMA user_version; PRAGMA integrity_check; SELECT kind, handle FROM artifacts;'

The output shows 5, ok, and the routes, LLM providers, LLM proxies, and MCP proxies on this gateway. Copy the upgraded database into the pod next to the old one, and compare checksums:

kubectl --context "$GATEWAY_CONTEXT" exec -i -n "$GW_NS" "$POD" -c gateway-controller -- \
  sh -c 'cat > /app/data/gateway.db.new' < "$GW_DIR/gateway.db"
shasum -a 256 "$GW_DIR/gateway.db"
kubectl --context "$GATEWAY_CONTEXT" exec -n "$GW_NS" "$POD" -c gateway-controller -- \
  sha256sum /app/data/gateway.db.new

If the two checksums differ, repeat the copy. When they match, replace the old database:

kubectl --context "$GATEWAY_CONTEXT" exec -n "$GW_NS" "$POD" -c gateway-controller -- \
  sh -c 'rm -f /app/data/gateway.db-wal /app/data/gateway.db-shm && mv /app/data/gateway.db.new /app/data/gateway.db'

Remove the override and redeploy the gateway, with no pause between the two commands. The operator redeploys an APIGateway only when its spec changes, so the second command changes spec.infrastructure.annotations. Until the redeploy replaces it, the 1.2.1 controller restarts and refuses the version 5 database without changing it.

kubectl --context "$GATEWAY_CONTEXT" patch -n "$GW_NS" "$GWC" --type=json -p \
  '[{"op":"remove","path":"/spec/template/spec/containers/0/command"}]'
kubectl --context "$GATEWAY_CONTEXT" patch apigateway "$GW_NAME" -n "$GW_NS" --type=merge -p \
  "{\"spec\":{\"infrastructure\":{\"annotations\":{\"upgrade.amp.wso2.com/gateway-version\":\"$GW_VERSION\"}}}}"

Wait until the gateway reports the new generation as programmed and the new controller is rolled out:

for i in $(seq 120); do
  s=$(kubectl --context "$GATEWAY_CONTEXT" get apigateway "$GW_NAME" -n "$GW_NS" -o jsonpath=\
'{.metadata.generation} {.status.conditions[?(@.type=="Programmed")].observedGeneration} {.status.conditions[?(@.type=="Programmed")].status}')
  echo "$s"
  echo "$s" | awk '$1==$2 && $3=="True"{ok=1} END{exit !ok}' && break
  sleep 5
done
kubectl --context "$GATEWAY_CONTEXT" rollout status -n "$GW_NS" "$GWC" --timeout=300s

The loop gives up after 10 minutes. If its last line is not two matching numbers and True, see Troubleshooting. Otherwise, check the controller:

kubectl --context "$GATEWAY_CONTEXT" get pods -n "$GW_NS" -o jsonpath='{..image}' | tr ' ' '\n' | sort -u
kubectl --context "$GATEWAY_CONTEXT" logs -n "$GW_NS" "$GWC" \
  | grep -E 'Migrating database schema|Database schema migrated successfully|Database schema up to date|Loaded configurations|Loaded API keys|Control plane connection established'

The images are $GW_REGISTRY/gateway-controller:$GW_IMAGE_VERSION and $GW_REGISTRY/gateway-runtime:$GW_IMAGE_VERSION. The logs show Migrating database schema from version 5 to version 6, Database schema migrated successfully, and Database schema up to date with version 6. The configuration and API key counts match the counts before the upgrade. Invoke one agent through this gateway before you start the next gateway.

Step 8: Verify existing workloads

Check	Command or action	Expected result
Agent Manager images	`kubectl get pods -n "$AMP_NS" -o jsonpath='{..image}' \| tr ' ' '\n' \| sort -u`	The `amp-*` images use the target version.
Migration history	`select max(id) from migration_history`	`0050`
OpenChoreo planes	`helm list -A`	All four planes use chart 1.2.8.
OpenChoreo authorization	List projects in the Console.	Existing projects appear. An empty list with no errors means the `client_id` patch is missing.
Gateways	`kubectl get apigateways -A`	Each gateway is `Programmed=True` at its current generation.
Console gateway status	In the Console left sidebar, under INFRASTRUCTURE, click Gateways.	Each gateway is listed and active.
Invocation	Invoke an agent that uses an LLM Service Provider or MCP Proxy, with an existing API key.	The call succeeds through the gateway.
Deployment and promotion	Redeploy an existing agent and promote it.	Both succeed.
Observability and evaluation	Open a pre-upgrade trace and run a monitor.	Historical traces load, new traces arrive, and the monitor completes.

Do not mark the upgrade complete because Helm reported success.

Troubleshooting

Symptom	Cause and action
The controller crash-loops with `unsupported schema version 4, expected 6`	The gateway redeployed before its database was upgraded. Do not delete the database. Run Step 7 from the command override onwards: the override stops the crash loop, and the in-place upgrade makes the database readable.
Gateway images stay at 1.2.1	The operator did not redeploy the gateway. Check `helm get values gateway-operator -n "$DP_NS"` for both image tags, then repeat the Step 7 patch with a new annotation value.
Agent invocations return `404 {"error":"Not Found"}`, which the Console reports as a CORS error, or an agent's LLM calls return `404`	The controller started with an empty database, so it has no routes, LLM providers, LLM proxies, or MCP proxies. Restore the gateway's Step 2 backup into its controller and upgrade it in place as in Step 7.
Agent invocations return `401 Valid API key required` after a restore	The API key was created or regenerated after the backup you restored. Regenerate the key in the Console.
APIGateway reports `Multi-Attach error for volume`	The old and new controller pods are on different nodes. Check that the controller Deployment uses the `Recreate` strategy, and delete the old controller pod.
Migration job fails	Read the job logs before the 300-second cleanup. Fix the cause and rerun `helm upgrade amp`. The database stays at `0043`.
Agent Manager images remain at v1.0.0	An explicit image tag override in your values pins the old version. Remove it and rerun the upgrade.
Helm fails with `401 Unauthorized`	Helm is not signed in to `registry.wso2.com`. Run the `helm registry login` command from Step 1 again.
Agent Manager pods report `ImagePullBackOff` for an image under `registry.wso2.com/<project>/` without `sts`	The release kept the v1.0.0 `global.ampImageRegistry`. Once the failed upgrade finishes, rerun it with `"${AMP_IMAGE_ARGS[@]}"`.
An agent's LLM calls return `401` through an LLM proxy created after the upgrade	The new LLM proxy expects the API key header set in its LLM Service Provider, and the agent sends `API-Key`. Send the header that the agent's LLM configuration shows in the Console.
Pods report `ImagePullBackOff`	The pod's namespace lacks `wso2-registry-credentials` (Agent Manager images) or `wso2-gateway-registry-credentials` (gateway images). Create it as in Step 1. For agent workloads, check the node's registry credentials (see Prerequisites).
An APIGateway stays `Programmed=False` and the operator cannot pull the gateway chart	The operator namespace lacks `wso2-gateway-registry-credentials`, or the Secret has no `username` and `password` keys. Recreate it as in Step 1.
OpenChoreo calls return empty lists or `403`	The plane upgrade reverted the `client_id` entitlement patch. Re-apply it as described in Upgrade platform dependencies.

Recovery to v1.0.0

There is no chart-only downgrade after the database migration. Keep the installation in maintenance mode.

Stop the Agent Manager API by scaling its Deployment to zero.

Restore the database:

export PGPW="$(kubectl get secret amp-postgresql -n "$AMP_NS" -o jsonpath='{.data.password}' | base64 -d)"
kubectl exec -i -n "$AMP_NS" amp-postgresql-0 -- env PGPASSWORD="$PGPW" \
  pg_restore -U agentmanager -d agentmanager --clean --if-exists \
  < upgrade-backup/ams-1.0.0.dump

Roll back the Agent Manager and extension releases with helm rollback. The gateway extension's bootstrap job also runs on rollback and needs Agent Manager to be reachable. If it fails, rerun the rollback.
Roll back the gateway-operator release to its revision before Step 6 (helm history gateway-operator -n "$DP_NS"). This restores chart 0.11.0 and all its v1.0.0 values, including the gateway chart 1.2.2 and images 1.2.1 from ghcr.io, and removes any registry values set in Step 6. For each gateway, restore the controller data from its Step 2 backup and redeploy it. A controller database that gateway 2026.10.0 migrated to version 6 cannot be used with 1.2.1.
Restore OpenChoreo and Thunder as described in Upgrade platform dependencies.

Restoring the database discards writes made after the backup. Repeat the Step 8 checks before you resume service.

Breaking changes​

Prerequisites​

Step 1: Set variables and record the installation​

Sign in and create the pull Secrets​

Step 2: Back up​

Step 3: Upgrade OpenChoreo, the observability modules, and Thunder​

Step 4: Upgrade platform resources, then Agent Manager​

Step 5: Upgrade the observability and evaluation extensions​

Step 6: Upgrade the gateway extensions and the Gateway Operator​

Step 7: Upgrade each controller database in place and redeploy its gateway​

Step 8: Verify existing workloads​

Troubleshooting​

Recovery to v1.0.0​