AI Agents in Production: Identity, Budgets, and Blast Radius

A demo agent has one user, one task, and a forgiving audience. A production agent has a queue, a bill, a security boundary, and a hostile network. The three signals I keep running into — sites actively blocking agents, research pipelines fanning out thousands of agents at once, and new filesystems built specifically for coding agents — are all symptoms of the same shift: agents are now network citizens with state, cost, and blast radius.

Here is the production view I wish I'd had before shipping my first one.

The Verge's piece on sites closing their doors to agents is not a policy story, it is an infrastructure story. Cloudflare bot management, robots.txt enforcement, pay-per-crawl, and login walls are all live. Your agent will get 403, 429, and a CAPTCHA where the demo got HTML.

If you build on other people's content, assume the rules can change weekly. A tool that returns "blocked" is a valid result your loop must handle, not an exception to swallow.

OpenAI's biological-data work with thousands of agents is impressive, but the engineering is unglamorous: a work queue, idempotency, deduplication, and a budget per unit of work. The same applies if you fan out 50 agents over your own backlog.

Temporal, Celery, and SQS all work here. The choice matters less than the invariant: every unit of work can be retried safely and has a hard cost ceiling.

Coding agents are the clearest case. Context windows are a bad database. A filesystem gives you durable state, diffable artifacts, and tools the agent already understands: rg, git, pytest.

Projects like LayerFS are interesting because they treat the agent's workspace as a first-class primitive rather than a side effect of a chat transcript. You can get most of the benefit without a new product:

# One isolated worktree per agent, then throw it away. Cheaper than merge conflicts. git worktree add /tmp/agent- $RUN_ID -b agent/ $RUN_ID origin/main cd /tmp/agent- $RUN_ID && pytest -q # review: git -C /tmp/agent-$RUN_ID diff origin/main Enter fullscreen mode Exit fullscreen mode A bounded loop you can actually ship The single most useful thing I've added to agent code is a hard stop: step limit, token budget, and per-tool error handling that feeds failures back into the loop instead of crashing it. This is complete enough to run, with your own tools swapped in.

# agent_runner.py — bounded agent loop with a token budget and error feedback. # pip install openai; export OPENAI_API_KEY=... import json, uuid from openai import OpenAI client = OpenAI () MAX_STEPS = 12 TOKEN_BUDGET = 40_000 def run_tool ( name: str, args: dict ) -> dict: # Replace with real tools. Keep them idempotent where side effects exist. if name == " lookup_order ": return { " order_id ": args [ " order_id " ], " status ": " shipped " } raise ValueError ( f " unknown tool: { name } " ) def run ( task: str, tools: list, run_id: str | None = None ) -> dict: run_id = run_id or str ( uuid. uuid4 ()) # caller supplies this for idempotency messages = [{ " role ": " user ", " content ": task }] used = 0 for step in range ( MAX_STEPS ): resp = client. chat. completions. create ( model = " gpt-4o-mini ", messages = messages, tools = tools, tool_choice = " auto ", timeout = 60, ) used += resp. usage. total_tokens if used > TOKEN_BUDGET: raise RuntimeError ( f " { run_id }: budget exceeded at step { step } ( { used } ) " ) msg = resp. choices [ 0 ]. message messages. append ( msg. model_dump ( exclude_none = True )) if not msg. tool_calls: return { " run_id ": run_id, " output ": msg. content, " tokens ": used } for call in msg. tool_calls: try: result = run_tool ( call. function. name, json. loads ( call. function. arguments )) payload = { " ok ": True, " result ": result } except Exception as e: # errors are data, not crashes payload = { " ok ": False, " error ": str ( e )} messages. append ({ " role ": " tool ", " tool_call_id ": call. id, " content ": json. dumps ( payload ), }) raise RuntimeError ( f " { run_id }: no final answer after { MAX_STEPS } steps " ) Enter fullscreen mode Exit fullscreen mode Two details matter more than they look. The run_id comes from the caller, so a retried job reuses the same key.

Tool errors are returned to the model as JSON, which is usually enough to recover; a thrown exception just kills a run that was one step from finishing.

Logging the final string tells you nothing when quality drops. Emit one structured event per step: model name, prompt template version, step index, tool name, arguments hash, latency, token count, and outcome. OpenTelemetry spans work; Langfuse, LangSmith, and Arize Phoenix are built for this shape of data.