The AI agent demo worked. Here is what breaks between week one and week three.

Every AI agent demo works. You wire up a model, point it at a handful of documents, and it answers questions in a way that makes the room nod. Then it goes near real work: real enquiries, real payments, real customers, and it starts failing in ways the demo never showed. I build AI agents for businesses through neu Intelligence, and the gap between demo and dependable is where almost all of the actual engineering lives. This is the checklist I wish more teams ran before they call a prototype a product.

Demos get a curated folder of perfect PDFs. Production gets the CRM with duplicate contacts, the spreadsheet two people edit at once, and the WhatsApp thread where the real booking details live. Before touching prompts, map every system the agent must read from and write to, and decide which one is authoritative for each fact. If two systems disagree and the agent has no rule to pick, you have built a confident rumor machine.

An agent should have a written contract: the tasks it owns end to end, the systems it may touch, the actions it may take alone, and the ones that always wait for a person. "Handle parent enquiries" is a feature. "Draft replies to enquiry messages; never quote prices outside this list; hand anything about refunds to a human" is an operable role. Most agent failures described as hallucinations are really scope failures: nobody wrote down what was allowed.

The safest first release is one where the agent does everything except send. It drafts, a human approves, and you log both. Two or three weeks of that gives you a measured error rate on real inputs, which is the only honest basis for widening permissions. Least privilege stops being a guess and becomes a dial you turn against evidence.

Sooner or later the agent meets something it has never seen. What happens then is not an edge case, it is the core design decision: does it freeze, guess, or hand off cleanly with the full context attached? Design the hand-back first, then the autonomy. An agent with a good escalation path is production ready long before a fully autonomous one.

One pilot query costs nothing. The interesting number is cost per completed task as volume grows: retries, long contexts, and chatty tool loops scale badly if nobody designed for them. Measure per task from week one, and treat cost reduction as a standing feature rather than an emergency project in month four.

When the agent misbehaves at 9pm on a Sunday, can someone read its logs, edit its prompts, and change its tools without a vendor's permission? Code, prompts, schemas and integration details should live in systems you control, with a runbook that ships with the build. Ask any agency what you keep when the contract ends; the answer tells you what you are actually buying.

None of this is exotic. It is the same discipline operations engineers have applied to services for decades: clear boundaries, observability, gradual rollout, an owner. The models are new; the reliability playbook is not. If you are evaluating an agent build: I build these at https://neu.ie (disclosure), and the assess, prototype, build, run sequence we use is written up at https://neu.ie/process/. The short version: diagnose the real operation before quoting anything, and let week one produce a working session on your actual workflow, not a requirements document.