ai newsDo., 5. Nov.
EilmeldungClaude Reads Screenshots at 2,576px Now. The Same PNG Can Cost 3x the Tokens

AI Agent Deadline Propagation: Stop Expired Runs From Burning Tokens

An agent can miss a 20-second user deadline, keep calling tools for another minute, and still return nothing useful. The visible failure looks like a timeout. The expensive failure is that every nested retry and sub-agent received a fresh timeout, so the run had no shared idea of when to stop. AI a

dev.toDonnerstag 04:35 · 05.11.9 Min. Lesezeit
Bild: dev.to

An agent can miss a 20-second user deadline, keep calling tools for another minute, and still return nothing useful. The visible failure looks like a timeout. The expensive failure is that every nested retry and sub-agent received a fresh timeout, so the run had no shared idea of when to stop.

AI agent deadline propagation fixes that. Give the whole request one absolute finish time, pass the remaining budget to every child operation, and cancel work that no longer fits. This is not a prompt instruction. It is runtime control that prevents a late answer from consuming tokens, workers, and downstream API capacity after the user has already moved on.

This guide shows a vendor-neutral pattern for interactive agent workflows. It works whether the model call, retrieval step, and tools come from one SDK or several.

Consider an agent with a 25-second product SLA:

Those settings do not add up to a 25-second workflow. They can create a chain that runs well past it. A tool can start at second 18, use its full 15 seconds, then hand a result to a retrying model call. If the user disconnects at second 25, the infrastructure may still be working and billing.

The fix is an absolute deadline, not a collection of relative timeouts. Every operation asks one question before it begins: “How much of the parent request remains?” If its realistic minimum time does not fit, it should take an approved fallback, queue asynchronous work, or stop with a useful partial result.

These controls complement each other. Treating one as another is the source of most confusing timeout behavior.

An LLM cannot reliably enforce these on its own. It does not own the wall clock, network sockets, queue leases, or subprocesses. Your runtime must calculate and enforce them at each boundary.

Use a deadline timestamp, not “25 seconds remaining.” A timestamp is stable as a request crosses functions and services. Each boundary calculates the remaining time again. Keep product policy separate from the user-provided prompt; a caller must not be able to extend a deadline by changing a message field.

type RunBudget = { runId: string; deadlineAt: number; // Unix milliseconds, set only at the trusted edge responseReserveMs: number; // time retained to serialize and return a result signal: AbortSignal; }; function remainingMs ( budget: RunBudget ): number { return Math. max ( 0, budget. deadlineAt - Date. now ()); } function usableMs ( budget: RunBudget ): number { return Math. max ( 0, remainingMs ( budget ) - budget. responseReserveMs ); } function assertTimeFor ( budget: RunBudget, minimumMs: number ) { if ( usableMs ( budget ) < minimumMs ) { throw new Error ( " DEADLINE_TOO_CLOSE " ); } } Enter fullscreen mode Exit fullscreen mode At the edge, choose the deadline from the interaction contract. A chat question may get 20 seconds. An internal research task may get two minutes.

A nightly job should usually use a job lease and its own completion deadline, not inherit a browser request deadline that expired while it sat in a queue.

The response reserve matters. Without it, a workflow may complete a model call at the exact deadline and then fail while validating JSON, writing headers, or sending the answer. Reserve enough time for the final validation and response path based on measured latency; start conservatively and tune it from traces.

A child needs the parent deadline, a local cap, and a little cleanup time. Passing the full original timeout at every layer defeats propagation.

async function withBudget < T > ( parent: RunBudget, localCapMs: number, minimumUsefulMs: number, work: ( signal: AbortSignal, timeoutMs: number ) => Promise < T >, ): Promise < T > { assertTimeFor ( parent, minimumUsefulMs ); const timeoutMs = Math. min ( usableMs ( parent ), localCapMs ); const controller = new AbortController (); const timer = setTimeout (() => controller. abort ( " child deadline " ), timeoutMs ); const onParentAbort = () => controller. abort ( parent. signal. reason ); parent. signal. addEventListener ( " abort ", onParentAbort, { once: true }); try { return await work ( controller. signal, timeoutMs ); } finally { clearTimeout ( timer ); parent. signal. removeEventListener ( " abort ", onParentAbort ); } } Enter fullscreen mode Exit fullscreen mode Use the signal in the actual SDK call. Creating an AbortController that no client observes only makes logs look controlled. For dependencies that cannot be cancelled, stop waiting at the deadline, mark the result as potentially in flight, and avoid treating an uncertain write as safe to replay.

Here is a tool call with a 4-second cap. If only 1.5 seconds remain after the response reserve, the tool never starts.

const facts = await withBudget ( budget, 4 _000, 2 _000, ( signal, timeoutMs ) => searchDocuments ({ query, signal, timeoutMs }), ); Enter fullscreen mode Exit fullscreen mode For parallel work, children share the same absolute deadline; do not divide it equally by default. Two fast calls can both finish within the same window. But the parent must abort stragglers when it has enough evidence or when the deadline arrives.

A retry is an extra attempt inside the same deadline, not permission to start a new timer. Retry only errors that are transient and operations that are safe to repeat. A timeout after a customer-facing write is not proof that nothing happened.

async function retryRead < T > ( budget: RunBudget, action: () => Promise < T > ) { const waits = [ 150, 350 ]; for ( let attempt = 0; attempt <= waits. length; attempt ++ ) { try { assertTimeFor ( budget, 500 ); return await action (); } catch ( error ) { const hasAnotherAttempt = attempt < waits. length; const wait = waits [ attempt ]?? 0; if (! hasAnotherAttempt || usableMs ( budget ) < wait + 500 ) throw error; await new Promise ( resolve => setTimeout ( resolve, wait )); } } throw new Error ( " unreachable " ); } Enter fullscreen mode Exit fullscreen mode For writes, use an idempotency key and read the system of record before retrying an unknown outcome. If the write might have succeeded, switch from “retry” to “reconcile.” If a human decision or a different permission is required, pause the run.

A deadline policy should never turn uncertainty into a duplicate email, charge, or record update.

An expired deadline does not have to mean an empty screen. Design the final states before you add tools:

For example, an analyst agent can return the three verified facts it found and omit an optional comparison that would take another tool round-trip. That is better than silently running long after the caller has left. Do not expose internal token counts or raw errors; say what completed, what did not, and what the user can do next.

Sub-agents need a deadline and cancellation signal in their task envelope. Pass only trusted control data, not a free-form prompt asking them to “finish quickly.” At every handoff, log the received remaining budget, selected local cap, and cancellation outcome.

type AgentTask = { runId: string; parentRunId: string; deadlineAt: number; task: { kind: " retrieve " | " summarize "; input: unknown }; }; function canAdmit ( task: AgentTask, estimatedMs: number ) { return task. deadlineAt - Date. now () > estimatedMs; } Enter fullscreen mode Exit fullscreen mode Queued work is different. A queue might delay a task until its interactive deadline is useless. On dequeue, reject an already-expired interactive task and record why. If the product supports background completion, create a new durable job with a new, explicit deadline and notify the user through the chosen channel. Never let a stale web request turn into unbounded background work by accident.

Deadline propagation is easy to describe and easy to break during a refactor. Add tests that control the clock and assert observable behavior.

Track deadline-expired runs by workflow and stage, remaining time at each tool start, cancellation lag, skipped optional work, and useful partial-response rate. These measures expose a slow dependency or unrealistic product budget. They are more useful than an average latency graph that hides the tail where users actually give up.

Start in observe-only mode. Attach a deadline to each run, log what would have been skipped or cancelled, and compare it with user outcomes. Next, enforce child timeouts for read-only tools. Then propagate cancellation to sub-agents and optional parallel branches. Introduce async handoff only when the product can clearly show job status and deliver completion.

Finally, review every write path separately. A deadline expiring around a mutation is a recovery-design question, not a latency tweak. Confirm idempotency, reconciliation, and who can resume the run before enabling automatic retries.

The central rule is simple: time belongs to the whole request. When every component spends from the same deadline, an agent can degrade gracefully, stop honestly, and leave enough evidence to recover safely instead of quietly burning resources after its work has stopped mattering.

AI agent deadline propagation passes one absolute completion time from the request boundary to every model call, tool, retry, and sub-agent. Each child uses only the time remaining, rather than starting a fresh full timeout.

A timeout limits one operation for a relative duration. A deadline is a fixed point in time for the whole workflow. A child tool might have a three-second timeout, but it should receive less if the parent has only one second left.

Interactive tasks should. Background jobs need a separate job-level deadline or lease that reflects their delivery promise. Do not carry a browser request deadline into a queued job that cannot start until later.

Only when the retry, expected response, and response reserve fit inside the remaining time—and only when the action is safe to repeat. For an unknown write outcome, reconcile the system of record before another attempt.

Stop awaiting the result at the deadline, record that the operation may still be in flight, and avoid blind replay of any external effect. Prefer SDKs and tools that accept cancellation signals for future work.

Measure the p95 time needed for output validation, serialization, and network response, then reserve at least that amount plus a small safety margin. Revisit it as the response path changes.

Quelle: dev.to (dev.to). Original öffnen