AI Agent Tool Calls Need Idempotency Keys, Not Retry Loops

TL;DR — Agent orchestration frameworks borrow retry-with-backoff from distributed systems but skip the part that makes retries safe: idempotency. When a tool call times out, the agent can't tell whether the action happened, and it 'fixes' the ambiguity by trying again — which is fine for a GET request and catastrophic for a refund, an email send, or a ticket creation. The fix isn't smarter prompting; it's borrowing idempotency keys and saga-style compensation from distributed systems and making them first-class parts of the tool schema.

Every agent framework has a retry loop somewhere. A tool call fails, times out, or returns something the model doesn't like, and the orchestrator does the obvious thing: it tries again. Sometimes with backoff, sometimes with a reflection step where the model reasons about what went wrong. It looks like engineering discipline. It's actually a known distributed systems bug wearing a trench coat.

Distributed systems solved the "should I retry this" problem decades ago, and the solution was never just backoff. Backoff controls when you retry. The thing that makes retrying safe is idempotency — a guarantee that doing the operation twice has the same effect as doing it once. Payment APIs have idempotency keys. Message queues have deduplication IDs. Distributed transactions have two-phase commit or, more commonly now, sagas with explicit compensating actions. None of this is optional plumbing. It's the actual mechanism that lets a retry loop exist without becoming a duplication machine.

Agent tool-calling stacks skipped straight to the retry loop and left the idempotency layer at home.

Here's the specific failure. An agent calls a tool — send_email, create_ticket, issue_refund, place_order. The call times out before a response comes back. From the orchestrator's point of view, this is undecidable: the action may have completed on the far side and the response just got lost, or it may never have executed at all. A TCP timeout tells you nothing about which.

A classic distributed system treats this as first-class uncertainty and engineers around it. An agent loop treats it as a text problem. The tool result comes back as an error-shaped observation, the model reads "the tool failed," and because the planning loop's job is to make progress toward the goal, the obvious next step is: try again. The model isn't being sloppy. It's doing exactly what a ReAct-style loop asks it to do — observe, reason, act — using the only information it has, which is a string that says something didn't work. It has no mechanism to distinguish "this never ran" from "this ran and I never heard back."

Run that loop against a refund API or an email send and you get duplicate refunds and duplicate emails. Run it against a ticketing system and you get three tickets for one complaint. These aren't exotic edge cases. They're the default behavior of any agent whose tool-calling loop includes retries on a non-idempotent action, which describes most production agents doing anything beyond read-only lookups.

The instinct is to cap retries, add exponential backoff, or have the model "check first" before acting again. All of these reduce the frequency of the problem without touching the cause. A retry limit of two still gives you two side effects instead of one. A "check first" step only works if the tool exposes a reliable way to check — most internal APIs don't, because nobody designed them to be queried for a pending agent action's status. And asking the model to reason its way out of the ambiguity assumes the ambiguity is resolvable from inside the conversation, which it isn't. The information needed to resolve it — did the write actually commit — lives on the server side, not in the token stream.

The deeper issue is architectural: function-calling schemas describe what arguments a tool accepts, not what guarantees it provides. A tool spec says create_ticket(subject, body). It does not say whether calling it twice with the same arguments creates one ticket or two. The orchestrator has no way to know, the model has no way to know, and the retry loop fires blind.

The fix has existed for a long time, it just hasn't been ported into agent tooling yet.

Idempotency keys: every mutating tool call should accept a client-generated key tied to the specific attempted action, not the arguments. If the orchestrator retries, it sends the same key. The downstream service — or a thin idempotency layer in front of it — recognizes the key, returns the cached result of the first attempt, and performs no new side effect. This is exactly how payment processors avoid double-charging on network retries, and there's no reason a ticketing or email tool can't do the same.

Sagas and compensating actions: for actions that can't be made idempotent at the API level — things with external side effects like sending a physical notification — the orchestrator needs an explicit compensating action registered alongside the forward action. If a multi-step plan fails halfway through, the loop shouldn't retry forward blindly; it should be able to unwind what it already did. An agent that books a flight and then fails to book the hotel shouldn't retry the flight booking — it should have a cancel_flight compensation ready to invoke, the same way a saga orchestrator does when a step in a distributed transaction fails downstream.

Durable action logs: the orchestrator needs its own record, independent of the model's context window, of which tool calls were actually dispatched and with what key. The model's conversational memory is not a reliable source of truth for "did this already happen" — it's lossy, summarized, and trivially dropped on a context compaction. The dedup and audit logic belongs in infrastructure the agent can't accidentally forget.

Most agent frameworks right now model a tool call as a stateless function invocation: arguments in, result out, retry on failure. That model is fine for read-only tools and actively dangerous for anything that mutates state. The schema for a tool needs a declared idempotency class — safe to retry, retryable only with a key, or non-retryable without a compensating action — and the orchestrator needs to enforce that class rather than leaving the retry decision to prompt engineering.

This also changes how you evaluate agents. "Did the agent complete the task" is the wrong success metric if it hides "and it completed the task three times." Agent eval suites need an assertion for side-effect count, not just task completion — did exactly one refund get issued, not did a refund get issued.

None of this requires a smarter model. It requires admitting that a planning loop calling external tools is a distributed system, and distributed systems have a thirty-year-old answer to the exact failure mode agents are reinventing today. The reasoning layer gets all the attention. The retry semantics underneath it are where the money actually leaks.