Article: The Evolution of Agentic Harnesses: From Chains to Graph to Code

InfoQ Homepage Articles The Evolution of Agentic Harnesses: From Chains to Graph to Code

This evolution has not been abrupt, but incremental, shaped by the growing demands of real-world applications. Across this arc of change, three distinct paradigms have emerged over time, each reflecting a deeper understanding of how to constrain and operationalize non-deterministic intelligence into dependable software systems.

Initial agentic systems were built around simple, linear workflows, which we call "chains ". This approach was intuitive and fast, enabling a quick time-to-market. You strung together a sequence of prompts where the output of step one feeds into step two, and so on. The harness was little more than glue code for tool calls and conversational memory, and ended up treating the LLM like a black-box function: text in, text out, with some syntactic sugar around it.

At LinkedIn, for example, one of our early Generative AI agents used a fixed 3-step chain built using the LangChain framework:

Figure 1: Simple Chain Workflow with RAG (Source: created by author)

While relatively fast to build and deploy, this simplicity led to fragility. The workflow, such as answering a user's question via a series of tool calls over the LinkedIn Economic Graph, depended entirely on the textual fidelity of the preceding step. A single mistake upstream would cascade through and derail the entire flow because there was no built-in mechanism to systematically detect or recover from the error.

This approach epitomized the "best-effort" nature of these early agentic products. Building the initial 80% of the experience was relatively fast, but closing the final quality gap for production required extensive iteration through rigorous evaluations, prompt refinement, and sustained engineering investment.

As use cases for agentic systems grew more complex, the inherent limitations and fragility of linear chains became impossible to ignore. Developers responded by adopting graphs, the natural abstraction for introducing branching, conditional logic, parallelism, and retries into the workflow.

In this approach, graph-based harnesses use nodes to represent distinct tasks (such as LLM calls, tool invocations, or validations) and edges to govern the control flow. This shift from a linear structure to a graph mirrored the evolution seen in both data pipelines and traditional distributed systems.

The key innovation of graph-based systems was not merely adding branches but making control flow explicit and declarative. Rather than embedding logic implicitly in prompts or code, developers defined execution plans that could be easily inspected, modified, and reasoned about. This created a reasoning-loop control flow in which the LLM could leverage the graph structure to route and execute complex, long-running tasks, often in dynamic environments.

For many complex tasks, a single monolithic agent application was not tenable. Graph systems led to the emergence of multi-agent systems with a top-level intelligent orchestrator to manage task flow and dependencies. This transition to multiple agents required first-class protocols for establishing asynchronous agent-to-agent communication, like A2A and ACP (which eventually merged into A2A).

Figure 2: Representative Multi-agent Architecture (Source: created by author)

Given the cognitive limitations of these agents, "human in the loop" (HITL) support was essential. Agents were not fully autonomous; instead, they sought clarification, received feedback, or requested approvals from human operators at key decision points, balancing autonomy with control.

At LinkedIn, we implemented orchestration and inter-agent communication by reusing the existing LinkedIn messaging system to handle long-lived tasks, broken into message sequences, ensuring guaranteed "First In, First Out" (FIFO) delivery and seamless message history lookup. This foundation provided:

The graph-based workflow was implemented atop LangGraph and enabled several critical functions that improved system robustness and complexity management:

These agents also required advanced memory to learn from user interactions and support an increasingly personalized, adaptive, and useful experience over time. Building atop the distributed multi-agent architecture allowed us to model Cognitive Memory itself as another agent that application agents could integrate with.

These innovations enabled the launch and subsequent iterations of Hiring Assistant, LinkedIn’s first AI agent designed for recruiters. Recruiters can choose to delegate time-consuming tasks to Hiring Assistant, including finding candidates and assisting in applicant review, so that they can focus on the most strategic, people-centric parts of their job.

Over time, we realized that delivering a truly great agentic experience required more than a capable model. Our agent needed strong domain intelligence, grounded in a rich organizational context, and the ability to improve over time. This required memory to learn from interactions and support an increasingly personalized, adaptive, and useful experience for recruiters.

While graphs significantly expanded the complexity of tasks agents could handle compared to chains, they remained fundamentally bounded automations built around human-defined workflows. As these workflows and state spaces grew more complex, developers faced increasing pressure to explicitly encode recovery paths, validation logic, and edge-case handling. This resulted in brittle, overly intricate, and often difficult-to-maintain flow definitions and evaluation frameworks. Rigidity also made graph-based approaches ill-suited to tasks that could not be cleanly expressed as deterministic workflows.

The most recent leap in agentic systems has come from treating code, rather than prompts, as the primary medium of reasoning. Instead of simply asking the model to "think through" a complex problem in natural language, coding harnesses generate, execute, and refine code in a sandbox environment.

This eliminates the fundamental limitation of asking developers to define intricate workflows and instead allows agents to "find" solutions by generating code with verifiable outputs. These harnesses often come with built-in tools for planning, task delegation, memory access, and token management/compaction.

A natural language output from an LLM is probabilistic and, prior to subsequent validation steps, inherently untrustworthy. A piece of code, however, is deterministic. It either compiles, runs, and delivers the correct output, or it fails. This pass/fail clarity provides the tight feedback loops necessary to dramatically improve system reliability.

To ensure safety and correctness, this architecture relies on four core components:

At LinkedIn, we have used this approach to build an internal tool that allows developers, product managers, data scientists, and even business operations personnel to ask natural language analytical questions of an agent and receive answers grounded in aggregated platform data and insights. The agent operates within our existing privacy, access control, and security frameworks, enforcing role and purpose-based data access, layering automated governance review over outputs, and routing model interactions through approved, compliant infrastructure.

The agent translates the user query into a combination of declarative SQL and imperative Python code, executes it against sample data, validates the outputs against strict constraints (including privacy, visibility, and governance checks), and iterates on it in a loop until all correctness criteria are met.

Using a code-based approach has resulted in a quicker time-to-market, since we’ve avoided the laborious work of creating bespoke prompts, tools, and workflow graphs, and running evaluation cycles to fine-tune them. The final system is not only more reliable but also significantly more auditable and reproducible.

Regardless of what stage your agentic system is at, from simple chains to complex coding harnesses, certain foundational principles separate robust systems from fragile ones.

Not every problem justifies the operational complexity of graphs or coding harnesses. Selecting the right abstraction is ultimately an engineering tradeoff across task complexity, reliability requirements, explainability, latency, and cost. The fundamental principle is simple: Use the least complex system that reliably solves the problem.

Chains are often sufficient for narrow, linear tasks where a single pass or a limited sequence of prompts can produce acceptable outcomes. Examples include drafting customer support responses, summarizing documents, moderating content, or extracting structured data from predictable inputs. These systems typically offer the lowest latency and compute cost because they minimize model calls, orchestration overhead, and state management. For many production workloads, chains remain the most economically efficient option, especially when rapid iteration or large-scale deployment is required.

Graphs become appropriate when workflows require conditional branching, retries, tool invocation, or explicit state transitions. Real-world examples include multi-step customer service resolution, enterprise IT automation, compliance pipelines, or onboarding systems where different actions are triggered based on intermediate outputs. While graphs improve resilience and operational control, they introduce additional latency through orchestration layers and often require multiple LLM invocations, increasing both serving cost and system complexity. They represent a middle ground: more reliable than chains, but more expensive to design and operate.

Coding harnesses are best suited for domains where precision, verification, and recoverability are critical. Examples include software engineering agents, financial analysis systems, legal document validation, scientific workflows, and infrastructure automation, in which generated outputs can be tested, compiled, simulated, or formally validated. These systems frequently incur the highest compute and latency costs due to iterative reasoning loops, execution of external tools, and validation cycles. However, they provide the strongest guarantees in environments where correctness materially outweighs speed or cost.

As models improve and costs decline, the instinct to over-engineer remains a common failure mode. More sophisticated abstractions often increase compute spend, latency, and operational complexity faster than they improve business outcomes.

The real advantage comes from disciplined abstraction selection: deploying the simplest system that meets reliability, speed, and cost requirements. Strong engineering organizations do not win by maximizing architectural sophistication; they win by optimizing capability per dollar.

The rapid evolution of agentic systems has triggered a fundamental change in the role of the developer. We are witnessing a professional transition from the 'Prompt Designer' to the 'System Architect'. In the early days of chain-based LLMs, success often hinged on the artisanal skill of crafting the perfect prompt. However, as systems mature into complex graph and code harnesses, the focus has shifted to rigorous software engineering: designing execution flows, managing state and dependencies, and building robust validation and recovery mechanisms.

This change is rooted in a broader goal: moving from a "best-effort" mindset to one of "engineered reliability". Early systems tolerated unpredictability as an inherent constraint. Modern agentic harnesses demand better; they are engineered to aim for determinism wherever possible, use controlled nondeterminism only where necessary, and feature clear failure modes with defined recovery paths.

This progression marks the evolution of AI agents from bleeding-edge fringe experiments into first-class production-grade systems. The novelty is not in the components themselves, but in how they integrate probabilistic models into deterministic frameworks.

The challenge and the opportunity here are to treat these systems not as magical black boxes, but as software systems that demand the same rigor, design, and engineering excellence as any other critical infrastructure.