Claude's API Now Holds Your Agent's Conversation for 5 Minutes. Keep Your Own Copy

Posted on Nov 3 • Edited on Nov 3 • AI-assisted

Turn 200 of an agent loop on the Claude API used to look exactly like turn 1, only bigger. You uploaded the whole conversation again, and the API, in Anthropic's own words, "validates and tokenizes all of it again" ( Claude docs ).

On November 2, Anthropic put message threads into public beta on the Messages API ( release notes ). You send the full conversation once. After that you send only the new messages plus the id of the previous response, and the server rebuilds the rest.

The catch is on the same docs page. The thread is "best-effort", it expires 5 minutes after its last request, and Anthropic tells you to keep your own copy of every message. It's a cache sitting in front of your conversation, and your conversation still needs a home of its own.

The release notes for November 2 say long conversations and agent loops "can now send smaller requests and get a lower time to first token." You turn it on with the message-threads-2026-08-12 beta header and one new request field, thread ( docs ).

Anthropic's developer account summed it up as: "The API caches your conversation on the server between turns" ( @ClaudeDevs ). The company's launch post says that on requests of 500K to 1M tokens, median time to first token drops by about 35%. That number is Anthropic's, it covers very long sessions only, and I haven't measured it. The same post says threads are already on by default for most Claude Code users.

It works on the Claude API, Claude Platform on AWS, Claude Models on Google Cloud (Anthropic managed) and Microsoft Foundry. The docs say threads "are in beta on every model," with a supported list that runs from Claude Haiku 4.5 and Sonnet 4.5 up to Opus 5.5, Fable 5.1 and Mythos 5.1.

Without a thread, every turn re-sends the whole stack. With one, only the new page travels.

There are three kinds of request on a thread ( docs ). Create is your normal first request with "thread": {"type": "create"}. Continue sends only the new messages with "thread": {"type": "continue", "previous_message_id": "<id>"}. Replay is a fresh create with the full conversation, for when the API can't continue anymore.

The SDKs ship a helper that does the bookkeeping. This is the Python shape from Anthropic's example, trimmed:

import anthropic client = anthropic. Anthropic () # Creating the helper sends no request. thread = client. beta. messages. thread ( model = " claude-opus-5-5 ", max_tokens = 1024, cache_control = { " type ": " ephemeral " }, system = " You are a support agent. Keep replies short. ", ) # Turn 1 creates the thread and sends everything. thread. send ({ " role ": " user ", " content ": " I have an order question. " }) # Turn 2 sends only the new message. reply = thread. send ( { " role ": " user ", " content ": " How long does shipping take? " } ) Enter fullscreen mode Exit fullscreen mode If you build the requests yourself, the field rules are strict:

Three more rules will bite a hand-rolled loop. The API doesn't check your new messages against the stored ones, so a message you send again "can appear twice in the conversation" ( docs ). Each response id can be continued once, and a thread is append-only, so editing or branching an earlier turn means a new create request. On a stream, the id arrives in message_start, but you continue only after message_stop.

The docs say it in one line: "Threads do not change pricing or rate limits." The launch post adds that "Claude still reads the full conversation on every turn, so token usage stays the same." What shrinks is the upload and the work before the model runs.

Prompt caching stays your job. "A thread does not cache your prompt for you", so you still send a top-level cache_control or place breakpoints, and a thread request can carry at most 4 of them ( docs ). A fifth returns a 400 with thread_unsupported_request.

If you've used OpenAI's Responses API, the idea is familiar. Its previous_response_id also chains turns without resending history, and OpenAI bills "all previous input tokens for responses in the chain" too ( OpenAI docs ). The difference is lifetime. The same OpenAI guide says response objects are kept "30 days by default". Anthropic's docs say a thread lives in memory, never on durable storage, and is deleted "no more than 1 hour after its last use".

The 5-minute window restarts with every response the API stores ( docs ). An agent that waits on a slow test suite, or on a human approving a tool call, can come back to a dead thread.

When that happens, a continue request returns a 404 with thread_not_found, and the recovery section is blunt: "The response does not say why." The listed causes include expiry, a turn the API served but didn't store (such as a tool call cut off by max_tokens ), a response written by a fallback model, a different API key or workspace, and a first image added to a conversation that had none.

Here is the line I'd tape to the monitor: "No response, to a create or a continue request, says whether the API stored that turn" ( docs ). You learn it on the next request. So the design that works is the one the docs prescribe: keep the full conversation on your side, treat the thread as an optimization, and build the replay path on day one.

The server's copy runs on a timer. The copy in your own store is the one that survives.

The SDK helper replays once on thread_not_found, thread_already_continued and thread_continue_in_flight, and starts a new thread after a refusal ( docs ). But it keeps the conversation in memory only "while the helper object exists." If your agent runs as a queue worker, a serverless function or anything that restarts, persist the messages yourself. The SDK's beta tool runner also uses a thread by default, which you can switch off when you create it.

Some setups can't use threads at all ( docs ):

If your agent runs short sessions, the gain is small, because the cost of resending grows with conversation length. If your agent runs long sessions in one process, with steady turns, it's a beta worth trying behind a flag, with the unthreaded path kept as the fallback. The docs even give the helper an unsupported_request_behavior setting, continue_unthreaded, that drops to plain requests for conversations the API can't store ( docs ).

Will you move your agent loop onto threads, or does a 5-minute server-side copy add more failure modes than it removes? Tell me in the comments.

I build Hermes IDE, so weigh this with that in mind: it runs Claude Code, Codex and other CLI agents side by side, each in its own terminal and worktree, source-available (BSL 1.1) on macOS, Windows and Linux ( download · GitHub ).

A best-effort cache in front of a source of truth you own, with an expiry and a replay path, is one of the oldest trade-offs in system design, and threads are a fresh case of it. The System Design Pocket Guide: Interviews walks through 15 real designs where you make that same call about where state lives and what happens when the fast copy disappears.

Expand AIFix AIFix AIFix Follow AI tools, news and automation for small businesses and creators. Research-based guides at ainews.useaifix.com Location Islamabad, Pakistan Joined Oct 23, 2026 • Nov 3 Dropdown menu Copy link Hide Useful that you pointed out the 5-minute expiry and the replay path. For an agent that waits on a human or a slow tool, it seems worth testing what happens on a turn that arrives just after the thread expires, since that is where a replay would kick in.