# Kata Containers Isolation Tier Run agents in an environment under [Kata Containers](https://katacontainers.io/) — each agent boots in a lightweight QEMU/KVM virtual machine with its **own guest kernel**, so there is no shared kernel with the host at all. This is the strongest isolation tier, intended for the most untrusted or multi-tenant agent workloads. Read the [gVisor isolation tier guide](/agent-platform/docs/v2026.11.0/guides/isolation-tiers/gvisor/.md) first if you haven't — Kata uses the same environment-level mechanism and the same dedicated-node model; this page covers what is Kata-specific. Hard requirement: KVM (nested virtualization) Kata boots a real VM per agent, so the node **must expose `/dev/kvm`**: * ✅ Bare-metal Linux with VT-x (Intel) or AMD-V * ✅ A nested-virtualization-capable cloud VM — for example GCP **Intel N2** with `--enable-nested-virtualization`, or AWS EKS bare-metal (`*.metal`) instances * ❌ macOS / Colima — no nested virtualization; use gVisor locally instead * ❌ GCP E2, AMD N2D, ARM T2A machine types, and Container-Optimized OS (read-only filesystem) * ❌ k3d / kind **node containers** — even on a KVM-capable host, the Kata guest-agent handshake cannot complete inside a container-based node; Kata needs a real Linux node Verify on the node itself, not on your workstation. From a machine with `kubectl` access, `kubectl debug` runs the check on the node: ``` kubectl debug node/ -it --profile=sysadmin --image=busybox:1.36 -- \ chroot /host sh -c 'ls -l /dev/kvm; cat /sys/module/kvm_intel/parameters/nested' # expect: a /dev/kvm character device, then Y (Intel) ``` This leaves a completed `node-debugger--…` pod in the current namespace; delete it afterwards. With SSH access to the node, run the two commands there instead. ## How It Works[​](#how-it-works "Direct link to How It Works") Identical to gVisor, with one naming detail: the isolation tier is called `kata`, but the RuntimeClass (and containerd handler) it maps to is **`kata-qemu`** — that is the handler kata-deploy registers. ``` Environment (isolation tier: kata) └─ deploy / promote → runtimeClassName: kata-qemu └─ RuntimeClass "kata-qemu" scheduling → pod on the Kata node, QEMU/KVM VM + guest kernel ``` Like gVisor, Kata uses a **dedicated node** labeled `kata=true` and tainted `kata=true:NoSchedule`, so existing runc (and gVisor) agents are never touched — no downtime. ## Requirements[​](#requirements "Direct link to Requirements") | Requirement | Details | | ------------------------ | -------------------------------------------------------------------------------------------------- | | A new (empty) Linux node | Joined to the same Kubernetes cluster; dedicated to Kata agents | | Virtualization | `/dev/kvm` present and functional (see warning above) | | Architecture | x86\_64 | | OS | Ubuntu (or similar) with systemd + containerd — **not** Container-Optimized OS | | Access | `kubectl` and `helm` access to the cluster (the installer runs from your machine, not on the node) | ## Set Up a Kata Node[​](#set-up-a-kata-node "Direct link to Set Up a Kata Node") `${AMP_DIST}` The commands below run scripts and manifests from the release bundle. If you do not already have it extracted, fetch it first and point `AMP_DIST` at it — the same bundle the install guides use: ``` export VERSION="2026.11.0" curl -fsSL -O "https://github.com/wso2/agent-manager/releases/download/amp/v${VERSION}/wso2-agent-manager-${VERSION}.tar.gz" tar xzf "wso2-agent-manager-${VERSION}.tar.gz" export AMP_DIST="$PWD" ``` On a node-side step, extract it on that node: the installer script runs there, not on your workstation. ### Step 1: Add a new node to the cluster[​](#step-1-add-a-new-node-to-the-cluster "Direct link to Step 1: Add a new node to the cluster") Provision a KVM-capable Linux node matching the requirements above and join it to your cluster using your platform's standard mechanism. Leave it empty. Taint the node as early as possible If your platform lets you set taints at node-pool creation, apply `kata=true:NoSchedule` there. An untainted node starts accepting regular platform pods immediately. The installer restarts the node's container runtime; running pods normally survive that, but if the node has already picked up workloads you can `kubectl cordon` + `kubectl drain --ignore-daemonsets --delete-emptydir-data` it first. To share the node with other pods instead, leave it untainted and run the installer with `KATA_TAINT=false`. ### Step 2: Run the installer from a machine with kubectl and helm access[​](#step-2-run-the-installer-from-a-machine-with-kubectl-and-helm-access "Direct link to Step 2: Run the installer from a machine with kubectl and helm access") Unlike the gVisor installer (which runs on the node), the Kata installer runs from **any machine with `kubectl` and `helm` access** and does everything through the Kubernetes API: ``` KATA_NODES="" bash "${AMP_DIST}/deployments/setup/install-kata.sh" ``` What it does: 1. **Pre-flights `/dev/kvm`** on every target node (via a short-lived privileged pod) and aborts if any node lacks it. 2. Labels the node(s) `kata=true`. 3. Installs upstream **kata-deploy** from its Helm chart (kata-containers `KATA_VERSION`, default `4.2.0`), scoped to the labeled node(s) so your other nodes are never reconfigured. The chart wires the node's containerd itself on standard Kubernetes, k3s and RKE2; the installer detects which one from the node's kubelet version. Nodes set up by the earlier manifest-based installer are migrated on the first run. 4. Applies the `kata-qemu` RuntimeClass and boots a test pod under it on every target node. The installer reports success only once that pod has run in its own VM, with a kernel different from the node's. 5. Taints the node(s) `kata=true:NoSchedule`. Set `KATA_TAINT=false` to leave them schedulable for other pods. 6. Lets the Fluent Bit log DaemonSet tolerate the taint, so agent logs are collected. It checks the toleration stored in your `observability-logs-opensearch` Helm release and, if it is missing, sets the chart value `fluent-bit.tolerations` (a `helm upgrade --reuse-values` at the release's installed chart version). It patches the DaemonSet only when that release cannot be upgraded with Helm, and then prints the `helm upgrade` command to make it durable, because a patch is lost whenever the DaemonSet is recreated. Upgrading from the earlier guide Re-run the installer on a node set up with the earlier version of this guide; it replaces the manifest-based kata-deploy with the Helm release. On RKE2, the `/etc/containerd/config.toml` stub and the `config.toml.tmpl` that guide had you write can stay: kata-deploy 4.x writes its runtime entry where RKE2 reads it alongside that template. Verify ``` kubectl get runtimeclass kata-qemu kubectl -n kube-system rollout status daemonset/kata-deploy kubectl get node --show-labels # kata=true ``` Keep the Fluent Bit toleration on later upgrades The toleration is stored in the `observability-logs-opensearch` release's values, so it must be present on every upgrade that re-renders the Fluent Bit DaemonSet. An upgrade with `--reset-values`, or one that passes new `--set`/`-f` values without `--reuse-values`, drops it unless you pass `--set "fluent-bit.tolerations[0].operator=Exists"` again. Agents on the Kata node then stop producing logs with no error. ### Local development[​](#local-development "Direct link to Local development") There is no macOS-local path for Kata (no nested virtualization, and k3d node containers cannot boot Kata VMs). On a KVM-capable **Linux** host running the repository's dev k3d cluster: ``` make setup-kata ``` This joins the host itself to the cluster as a real (non-Docker) k3s agent node and then runs `install-kata.sh` against it. ## Create a Kata Environment[​](#create-a-kata-environment "Direct link to Create a Kata Environment") Isolation tier is set at environment creation time. Open the **Create Environment** drawer in the console (Environments → Create Environment), select **Sandboxing T3 (Kata Containers)** from the **Isolation Tier** dropdown, and run the generated command — it includes `ISOLATION_TIER=kata` automatically. If you script environment creation directly, set the variable yourself: ``` ISOLATION_TIER=kata \ ENV_NAME=secure-prod \ DISPLAY_NAME="Secure Prod" \ AGENT_MANAGER_TOKEN= \ bash "${AMP_DIST}/deployments/scripts/add-environment.sh" ``` The tier appears as a **Sandboxing T3** shield chip in the environment list, and as a shield icon next to the environment name on the agent's Deploy and Overview pages — hover over it to confirm *Sandboxing Tier 3 — Kata Containers*. A new environment is not part of any deployment pipeline yet. Add it to your project's pipeline ([Managing Deployment Pipelines](/agent-platform/docs/v2026.11.0/guides/environment-management/.md#managing-deployment-pipelines)), then deploy or promote an agent to it as usual. ## Verify VM Isolation[​](#verify-vm-isolation "Direct link to Verify VM Isolation") A Kata agent runs a **different kernel** from the node — that's the definitive check: ``` kubectl get pod -n -o jsonpath='{.spec.runtimeClassName}' # kata-qemu kubectl exec -n -- uname -r # Differs from the node's own `uname -r` kubectl exec -n -- grep hypervisor /proc/cpuinfo # hypervisor flag present ``` If the pod reports the same kernel as the node, it is **not** running under Kata (the RuntimeClass wasn't applied or the pod fell back to runc). ## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting") Pod stuck in ContainerCreating Almost always missing nested virtualization. Confirm on the node itself (with the `kubectl debug` command under the KVM requirement above, or over SSH): ``` ls -l /dev/kvm cat /sys/module/kvm_intel/parameters/nested # expect: Y kubectl describe pod -n # look for kvm / RuntimeClass errors ``` No `/dev/kvm` → move to a bare-metal or nested-virt-enabled host (e.g. GCP Intel N2 with `--enable-nested-virtualization`). `no runtime handler kata-qemu` errors The RuntimeClass exists but containerd on the node has no `kata-qemu` runtime — the install didn't complete or containerd wasn't reloaded: ``` kubectl -n kube-system rollout status daemonset/kata-deploy kubectl -n kube-system logs -l name=kata-deploy kubectl get node -L katacontainers.io/kata-runtime # true once kata-deploy has wired the node ``` Re-running `install-kata.sh` is safe: it re-applies the kata-deploy chart and boots a test pod on every target node again, failing with the reason if Kata still cannot run there. Pod runs but no logs / metrics / traces / try-it This is cross-node networking or log-collection scheduling, not Kata itself — the failure modes and fixes are identical to gVisor's. Work through the [gVisor troubleshooting section](/agent-platform/docs/v2026.11.0/guides/isolation-tiers/gvisor/.md#troubleshooting), substituting `runtimeClassName: kata-qemu` in the test pod. Performance notes * Booting a VM adds startup latency compared to runc/gVisor; Agent Manager's warm pools absorb most of it. * `kubectl port-forward` should be avoided for Kata pods — reach agents through their Service/gateway endpoint. * Usage metrics are attributed to the sandbox VM as a whole (reported under a `sandbox` container label), including the guest kernel's footprint. ## Next Steps[​](#next-steps "Direct link to Next Steps") * Lighter-weight syscall isolation without KVM: [gVisor isolation tier](/agent-platform/docs/v2026.11.0/guides/isolation-tiers/gvisor/.md). * [Environment management](/agent-platform/docs/v2026.11.0/guides/environment-management/.md) — pipelines, promotion, and environment settings.