Loop vs Graph vs Harness: How AI Agents Actually Work

If you're starting to build AI agents, you'll hear three words a lot: loop, graph and harness.

They sound complicated. They're not. In this article I'll explain all three with sticky-note diagrams. No code needed yet: that comes in Part 2.

We'll use one example the whole way through:

A customer asks: "Where's my order #123? If it's late, I want a refund."

Every agent has to answer one question at every step:

There are two answers: the loop and the graph.

For our customer, the model might check the order, see it's 5 days late, check tracking, see it arrives today, and decide a refund isn't needed. You never wrote an if/else. The model chose the steps.

That's the strength of a loop, and also the risk: it could have refunded straight away.

👉 The model picks the path. You design the environment.

Same customer, but this time you draw the path:

Start → Look up order → Is it late? No → LLM writes a tracking reply → End Yes → LLM drafts a refund offer → Manager approves → End Enter fullscreen mode Exit fullscreen mode The model never decides to refund. Your code decides, and a manager must approve. The model only writes the friendly text inside the boxes.

👉 You pick the path. The model works inside the boxes.

In real systems you usually use both: a graph for the business process, with loops running inside the steps that need real reasoning.

Build the loop first. Then ask: "Which decision would I be nervous letting the AI make alone?" Turn that decision into a graph edge.

The model by itself can only take text in and give text out. It can't run tools, remember anything or stop itself. Something has to do all of that around it.

Think of a very smart new employee on day one:

Harness = orchestration (loop, graph or both) + tools + memory + rules + logs

A common shortcut is "the harness is the loop". That's too narrow. The loop or graph is the skeleton, but tools, context, memory and guardrails are where most of the real engineering happens. The same model in two different harnesses can behave like two completely different products.

Loop = tools + a circle + a stop sign Graph = state + boxes + arrows Agent = model + harness

Which approach are you using in your agents right now: loop, graph or both? Let me know in the comments 👇

Next in this series: we'll build this harness in real Java code with Spring AI, then look at who builds it for you on Azure.