What the AI Engineer World's Fair got right about agents (and what I'm still skeptical about)

I went to the AI Engineer World's Fair as a backend person who builds the plumbing agents run on, not the agents themselves. That made it a strange and useful conference, and this is my honest summary.

The state conversation. Almost every serious talk about agents ended up being about where state lives and how a failed step is retried. That's my day job, and it was refreshing to hear people treat it as the core problem instead of an implementation detail.

Evaluation. A team described running the same forty cases against every model change and every prompt change, and treating a regression there as a release blocker. That's just testing, but with a different surface area, and the teams doing it looked much less nervous.

Tool design. The good talks treated tool calls like API endpoints: explicit input schema, explicit error shapes, and a way to tell whether a call is safe to retry. That's the same discipline as any internal RPC, and it's encouraging to see it applied to something as fuzzy as an agent.

Orchestration frameworks that hide the control flow. If I can't read the code and say what happens after step three fails, I don't want it near a production system. A state machine with a clear transition table is less impressive in a demo and much easier to operate.

Benchmarks that don't measure the cost of failure. A model that's slightly better but occasionally does something irreversible is not better for us. The interesting metric is not accuracy, it's how often a mistake reaches the outside world.

A requirement that every tool call in our pipeline carries an idempotency key, and a rule that the agent loop writes each step's result before moving on. Both are old backend ideas. That was, oddly, the theme of the whole conference: the AI part is new, but the reliability part is the same as it has always been.