Idempotent Tool Design: Building Agent Tools Safe to Retry

An agent is halfway through a customer support task. It calls an issue_refund tool, the refund service responds slowly, and the tool call times out. The model sees an error, decides the step failed, and calls the tool again. Both calls succeed. The customer gets refunded twice, and the agent reports that everything went fine.

Timeouts are routine, and retrying is what agents do. The bug lives in the tool, which treated two attempts at one refund as two refunds. This guide covers why agent retries behave differently and which patterns keep a retry from becoming a second side effect. It expands one recommendation from the broader set of design changes APIs need for agent traffic. It's also part of our series on agent-ready APIs.

Why Agents Retry Differently Than Humans​

Picture a person paying for an order on a slow checkout page. The spinner keeps turning. Most people wait, refresh with care, or check their bank app before they press Pay again. They know a payment matters, so they treat the uncertainty with caution.

An agent has no such instinct. As Nuwan Dias, VP and Deputy CTO at WSO2, puts it, a payment is "just another step in a series of steps" to an agent. If that step runs long, the agent retries without weighing the consequences.

Agent retries also come from more places than a refresh button:

1. The client layer: HTTP client libraries can retry transient failures on their own. Stripe's engineering blog notes, for example, that its Ruby library retries failed requests automatically.
2. The model: After a timeout or an unclear error, the model can call the same tool again, sometimes with different arguments.
3. Recovery and resumption: An agent that crashes and resumes from a checkpoint can re-run tool calls it already completed. The authors of ACRFence call this action replay. They surveyed 12 agent frameworks and found tool-call side-effect issues across them, with duplicate payments among the consequences.

Fan-out makes this worse. WSO2's analysis of agent traffic describes one tool call fanning out into dozens of search, product detail, and inventory requests. Each one is another chance for a timeout and a retry. For human-driven clients, idempotency improves reliability. For any tool an agent can call, it's a requirement.

What Idempotency Actually Means (And What It Doesn't)​

An operation is idempotent when calling it several times with the same input leaves the system in the same state as calling it once. Duplicate requests can still reach the server. Idempotency makes those duplicates harmless, because a repeat has no additional intended effect. The server can still log each request. The response to a repeat can differ, but the outcome can't.

HTTP defines this property per method. Under RFC 9110, PUT, DELETE, and the safe methods such as GET are idempotent. POST isn't on that list, and PATCH isn't guaranteed to be idempotent either. Stripe's engineering post on idempotency shows why PUT is safe to repeat. Its example request carries the full definition of a DNS record. If the record already exists, the server ignores the duplicate and returns success.

Three things idempotency does not mean:

1. It doesn't mean identical responses: A repeated DELETE can return 404 where the first call returned 200. RFC 9110 allows the response to differ while the intended effect stays the same.
2. It doesn't come from the method name alone: A POST handler can be made idempotent with a key. A PUT handler can break the rule if it appends instead of replacing.
3. It doesn't assume byte-identical input: Agents rarely send exactly the same request twice. A retried tool call may reformat an amount or reword a note. Research on agent replay studies these semantically equivalent repeat calls directly. Don't fuzzy-match them to decide two financial operations are the same. Define a short list of allowed normalizations, such as one canonical format for amounts, and reject any other difference under the same key.

The Idempotency Key Pattern​

Some operations can't be idempotent by nature, such as charging a card or issuing a refund. An idempotency key makes them idempotent by contract. The client generates a unique key for one logical operation and sends it with the request. The server stores the key with the outcome. If the same key arrives again, the server returns the stored result instead of running the operation twice. Keys are only kept for a limited window, though. Stripe, for example, can remove keys once they're at least 24 hours old in its v1 API, so a retry after that point runs as new work.

Stripe frames the key's value around three failure points. Each one maps to something an agent's tool loop hits in production:

The third row is the one that produces duplicate refunds. It's also the one an agent can't detect, because a lost success and a real failure look the same to the caller.

The second row needs care too. An unknown outcome isn't proof that nothing happened. A worker can crash after the refund goes through but before it records the result. Decide which component owns recovery. If the write reaches an external provider, pass the same key downstream or reconcile against the provider's records before running anything again.

The server-side mechanics, such as atomic key claims, request fingerprints, and retention windows, are covered in our guide to idempotency keys for APIs. For agents, the harder question sits upstream: who creates the key, and when.

Create the key outside the model, once per logical step​

If the model writes the key, a repeat call can arrive with a fresh one. The server then treats it as new work. Instead, have the agent runtime bind the key to a durable operation identity. A plan step ID alone isn't enough, because replanning can reuse a step ID for different work. Tie the identity to the tenant or user, the tool, and the authorized parameters. Reuse it for every retry of that authorized action, and reject a call that reuses it with different parameters. Create a new identity only for a new authorized action.

import uuid

# key_store and tool.invoke are application abstractions, not MCP SDK methods.
def call_write_tool(operation_id: str, tool, args: dict):
    # The runtime owns the key. The model never generates it.
    # create_or_get is atomic and durable, for example an INSERT with a unique
    # constraint on operation_id that returns the existing row on conflict.
    # Every worker, including one that restarts, gets the first stored key.
    key = key_store.create_or_get(operation_id, default=str(uuid.uuid4()))
    return tool.invoke(args, headers={"Idempotency-Key": key})

Store the key before the first attempt. Otherwise, a crash between sending and storing leaves the resumed agent unable to recognize its own earlier call.

One operation, one key: how a retried tool call stays a single refund.

Beyond the Key: Idempotency as Part of a Tool's "Effect Type"​

A key prevents duplicate execution. It doesn't tell anyone which tools need one. That decision is easier to get right when every tool states its effects explicitly.

Engineer Cam Pedersen sketches one way to do this in his essay The Importance of Being Idempotent. He gives each tool call a type, and treats its effect profile as the part that matters. A refund tool, in his example, is irreversible and requires a key. A ticket lookup has no effect worth undoing. He also tracks taint, meaning whether a call's inputs may come from untrusted content.

Applied to a support agent's tools, an effect profile might look like this:

The reversibility classes come from Revisable by Design, a 2026 paper that sorts agent actions into idempotent, reversible, compensable, and irreversible. Pedersen uses "pure" where the paper says "idempotent." He also flags the combined signature as his own synthesis, so treat it as one emerging approach, not settled practice.

The takeaway holds either way. A refund and a ticket lookup carry very different risk, so idempotency is a per-tool decision. A blanket policy either over-engineers safe reads or under-protects dangerous writes.

MCP gives you a place to record part of this. The MCP specification defines optional tool annotations, including readOnlyHint, destructiveHint, and idempotentHint. It also says clients must treat annotations as untrusted unless they come from a trusted server. Annotations describe a tool. They don't enforce anything. Enforcement has to live in the runtime or the gateway, and so does scoping which agents can call irreversible actions at all.

Failure Modes: What Happens When Tools Aren't Idempotent​

Missing idempotency rarely shows up in a demo. It shows up under load, during an incident, or when a long task resumes. Three failure modes are worth separating, because each one needs a different fix.

Duplicate writes from naive retries​

This is the refund-twice scenario. Stripe's version is a charge endpoint: call it twice by accident, and the customer pays twice. With agents, the duplicate can arrive seconds later from the model or hours later from a resumed run.

Retry storms that deepen an outage​

This failure has nothing to do with duplicates. When a backend degrades, every agent session that depends on it starts retrying. Stripe notes that retries during an incident can contribute to further degradation. If many clients fail together and retry on similar schedules, their attempts arrive in waves. That's the thundering herd problem. It hits perfectly idempotent tools too. Idempotency stops the duplicate effect, but it does nothing about the extra load.

Plan-level replay​

This one is specific to agents. If an agent never learns that a step succeeded, it may mark the whole task as failed. Then the agent, or the user, starts the task again. Now every write in the plan can repeat, not just the one that timed out. One missed acknowledgment becomes a duplicate order, a duplicate email, and a duplicate refund. The fix has two parts: idempotent tools, plus keys tied to the task so that a resumed task reuses them. A genuinely new task should get new keys.

Plan-level replay is also the hardest to spot, because each request looks valid on its own. Finding it means grouping calls by task, which the gateway section below returns to.

Patterns for Safe Retry​

Idempotency keys are the baseline. The other four patterns here cover what keys can't.

• Idempotency keys on non-idempotent writes: Any write that isn't naturally idempotent should require a key, and the server should replay the stored result for repeats. A full replace, such as a PUT that sets an address, can work without one. Make that call per operation in its contract, not by guessing which writes are low-risk.

• Exponential backoff with jitter: Stripe recommends waiting in proportion to 2^n after the nth failure. It also recommends random jitter, so that clients don't retry in lockstep. For agents, apply this in the runtime, not in the prompt. Cap the number of attempts, and don't retry errors that won't change, such as validation or permission failures. Server-side resilience patterns, such as circuit breakers, cover the other half of this problem.

• Dry-run operations for irreversible actions: A POST /checkout-dry-run endpoint returns the final effect, such as the total price and delivery date, without committing anything. WSO2's design guidance uses it to close a gap agents create. A human reviews the final price before paying, but an agent tends to commit straight away.

• Two-step confirmation for high-stakes writes: Split "propose" from "commit." The first call creates a pending confirmation. Only a second, explicit call completes the order. This is an application-level confirmation flow, not distributed two-phase commit.

  POST /checkout
  -> 201 Created {"confirmation_id": "cnf_7Q2"}
  
  POST /confirm-checkout/cnf_7Q2
  -> 200 OK {"order_id": "ord_91"}
  

  The first call isn't always harmless. It can reserve stock or hold funds, so give it an idempotency key too. A retry then returns the same confirmation_id. Bind each confirmation to the actor, the operation, its parameters and amount, an expiry, and a single use. Consume it atomically when the order completes, and make retries of the confirm call idempotent as well.

• Bulk endpoints to shrink the retry surface: An agent that calls a single-item endpoint once per item creates many chances for partial failure. A batch endpoint such as POST /products-batch-get turns many calls into one. WSO2's guidance calls it likely one of the highest-return changes for agent traffic.

A Pre-Publish Checklist for Tool Authors​

Before you expose a tool to agents, run it through these questions. Each one is phrased as a test, so it has a clear pass or fail.

• Does every write that isn't naturally idempotent reject requests that arrive without an idempotency key?
• Does the agent runtime, not the model, create the key once per authorized operation, atomically, and store it before the first attempt?
• Does a repeat with the same key return the original result, including after a crash and resume?
• Do irreversible tools offer a dry-run or a two-step confirmation flow?
• Does the retry loop back off with jitter and stop on errors that won't change?
• Is each tool's effect profile written down, in its MCP annotations and in its API description? WSO2's design guidance proposes an x-ai-metadata block for this.
• Have you tested a semantically equivalent retry, such as the same refund with the amount formatted differently?

Making Idempotency a Platform Guarantee​

Everything above works only if every team remembers to do it. Once several teams ship agent-facing tools, the checklist gets applied unevenly. That's when it pays to move some checks to the gateway, which sees every call.

On WSO2 API Platform, the API Gateway handles REST API traffic and the AI Gateway handles MCP and LLM traffic. Both are governed from one control plane. On the AI Gateway, keep the two flows distinct. Tool calls go through an MCP proxy to the MCP server and its APIs. The agent's own model calls go through an LLM proxy to the provider. The controls below apply to the tool-call path.

• Per-tool authorization and visibility: The AI Gateway's MCP governance policies work at two different points. The MCP Authorization policy checks each tool call against JWT claims or OAuth scopes. An agent without the right scope for issue_refund gets a 403 when it calls the tool. The MCP Access Control List policy filters tools out of the tool list and also rejects direct calls to them. Use authorization to decide who can invoke a tool, and the access control list to decide which tools are exposed at all.
• Per-tool rate limits: The MCP Rate Limit policy counts calls per tool. That lets you cap a retry storm against one irreversible tool without throttling reads.
• Required idempotency keys: Gateway policies run as a chain on each request, and you can write your own. On REST traffic, a custom policy attached to a write operation, such as POST /refunds, can check for an Idempotency-Key header and reject requests that lack one. MCP traffic is different. Every tool call is an HTTP POST to the same endpoint, so a header check alone can't tell a read tool from a write tool. A policy would need to read the tool name from the JSON-RPC body, which custom policies can do in the request-body phase. The MCP server also has to pass the key on to the service that performs the write. Even then, a presence check only confirms that a key exists. Deduplication still belongs in the service that owns the write. Stateful rules, such as requiring a recent dry-run, need a shared store and a deliberate design.
• Retry visibility per task: Have the agent runtime send a task or session ID header with each call. Gateway logs and traces can then group calls by task. A rising count of retries per task for one tool is an early warning.

Central enforcement earns its keep in three situations:

• Several teams publish agent-facing tools.
• Those tools touch payments or regulated data.
• You need to attribute retries to a specific agent.

Before that point, a well-reviewed tool contract may be enough.

Conclusion​

Agents retry because networks fail, models are unsure, and long tasks resume. Idempotent tool design accepts that. Every non-idempotent write carries a key created by the runtime, every tool declares its effects, and irreversible actions get a dry-run or a confirmation step. A gateway then turns those rules from team habits into platform defaults. To see how the AI Gateway routes and governs MCP traffic, start with the AI Gateway documentation.

Frequently Asked Questions​

What does idempotency mean in API design?​

An idempotent API operation produces the same server-side outcome whether a client sends it once or several times. GET, PUT, and DELETE have this property by HTTP definition. A POST endpoint usually needs an idempotency key to get it.

What is an example of an idempotent operation?​

Setting a shipping address to a specific value is idempotent, because repeating it changes nothing. Adding 10 credits to an account is not, because each repeat adds 10 more.

Should AI agents retry failed tool calls automatically?​

Reads can be retried with backoff and a retry cap. Writes should be retried only when the tool is idempotent, or when the runtime reuses the same idempotency key for every attempt.

Do MCP tool annotations make a tool idempotent?​

No. The idempotentHint annotation only describes intended behavior, and the MCP specification tells clients to treat annotations as untrusted unless the server is trusted. The server still has to detect and absorb repeats.