Building Locdex: Exploring Local AI Models and the Architecture of Coding Agents

Understanding how local inference, repository intelligence, model routing, and execution controls can improve AI-assisted software development.

Artificial intelligence has become an increasingly important part of software development, with large language models (LLMs) enabling developers to generate code, debug applications, write tests, and perform other programming operations using natural language instructions. Tools such as GitHub Copilot, Cursor, and Claude Code have expanded these capabilities beyond code suggestions into agentic systems that can interact directly with software repositories.

AI coding agents differ from conventional code-generation assistants in their ability to execute development tasks through external tools. Rather than merely returning a function or code snippet, an agent can examine existing files, modify source code, execute commands, and verify the changes made to a project.

However, the development of autonomous coding agents introduces additional challenges, particularly when running smaller language models locally. Model capability, hardware limitations, repository size, context management, and execution security all influence the reliability of these systems.

When my cofounder and I began developing Locdex, our initial objective was to create an open-source AI coding assistant that could operate locally without requiring paid cloud inference for every development task, but as development progressed, we began exploring the underlying architecture required to make local coding agents more effective. This led us towards building a broader runtime that combines local model execution, repository intelligence, context management, controlled tools and optional cloud inference.

This article explores the architecture behind Locdex, the problems we're attempting to address, and how these systems interact during software development.

A local AI coding agent is a software system that uses a language model running on a developer's machine to understand and execute programming tasks.

Unlike cloud-based AI services, where inference takes place on remote infrastructure, local inference uses the developer's available computing resources, including the CPU, GPU and system memory.

Local models provide several advantages, including greater control over inference, reduced dependence on external APIs and the ability to operate without transmitting every prompt to a cloud provider.

However, these benefits come with limitations.

A developer running a 7-billion-parameter coding model on a laptop may experience different results from someone using a larger model on a high-memory workstation. Similarly, tasks involving multiple files, extensive dependencies or complicated debugging procedures may exceed the capabilities of smaller models.

The challenge is therefore not limited to making models run locally. It involves building an execution environment capable of helping those models understand repositories, interact with development tools and verify their results.

Locdex is an open-source coding-agent runtime designed to operate directly inside software repositories.

It provides an interface through which developers can instruct AI models to perform programming tasks, including inspecting files, modifying source code, running tests and examining Git changes.

The architecture is local-first, meaning developers can configure and use local coding models without making cloud inference a requirement. External models can also be configured when additional capability is required.

Locdex uses a Python-based runtime and provides a command-line interface (CLI) for interacting with projects.

The public version is available through PyPI, while a substantially expanded architecture is being developed separately.

For the public v1.3.1 release, installation begins with:

After installation, users can configure the runtime and selected model through:

The coding agent can then be started inside a repository using:

The newer development architecture introduces additional commands, persistent sessions, broader model selection and more detailed execution controls. These changes are being tested before becoming part of a stable release.

One of the major challenges associated with coding agents is understanding existing software repositories.

Modern applications often contain hundreds or thousands of files distributed across different directories, modules and dependencies. Sending an entire repository to a language model for every request is computationally expensive and frequently unnecessary.

Consider a developer working on an authentication bug in a Node.js application.

The application may contain frontend components, database configurations, authentication middleware, API routes, payment services and several other modules.

While the developer's request concerns authentication, much of the repository may be unrelated to the task.

Passing all these files into the model increases the amount of context processed without necessarily improving its ability to identify the problem.

To address this, we're developing two separate systems within Locdex: repository intelligence and context compilation.

Repository intelligence enables Locdex to examine the structure of a software project and identify relationships between files, functions, symbols and tests.

Rather than relying entirely on a model to discover relevant files through repeated tool calls, the runtime can retrieve useful information about the repository before constructing the model's context.

For example, a request to fix an authentication endpoint may require identifying the relevant controller, middleware, token validation function and associated tests.

The repository intelligence system is intended to make these relationships easier to retrieve.

Context compilation determines which repository information should be included in a model request.

Although repository intelligence may contain extensive information about a project, the model does not necessarily need all that information to perform a particular task.

The context compiler therefore attempts to construct a smaller representation containing relevant source code, dependencies, task information and recent execution results.

This distinction is important because increasing context size can increase inference latency, memory requirements and cloud API costs.

Locdex's approach is based on providing the minimum sufficient context required to execute a task.

We're also working on persistent task states and context compaction to improve continuity during longer development sessions without repeatedly transmitting unnecessary information.

The effectiveness of these mechanisms still requires comparative testing across different repositories and coding models.

Local language models have different capabilities depending on their architecture, size, training and available computing resources.

A smaller model may perform adequately when modifying a simple function, while a more complicated task involving several modules might require a stronger model.

Running the strongest available model for every task can also be inefficient.

Model routing provides a mechanism for selecting an appropriate model based on the requirements of the task and available execution resources.

Within Locdex, the routing system considers task characteristics, hardware constraints, available models and user-defined policies.

For example, suppose a developer requests a correction to a multiplication function that returns the sum of two numbers.

A smaller model may be capable of identifying and correcting the error without requiring a larger cloud model.

However, a complex debugging task involving multiple services may require additional reasoning and a different execution path.

Our current routing architecture uses deterministic rules, with ongoing development towards models that can make decisions using historical task outcomes and comparative performance measurements.

The objective is to evaluate different models based on their probability of completing a task successfully, rather than selecting models solely because of their size or nominal inference cost.

An important consideration is that repeated failures also consume resources. A smaller model that repeatedly attempts an unsuccessful task may eventually cost more computational time than a stronger model that completes it earlier.

For this reason, verification results and task success rates are important inputs to future routing improvements.

AI coding agents require access to development tools to execute tasks beyond code generation.

These tools may include file readers, code editors, terminal commands, test runners and Git operations.

While tool access expands an agent's capabilities, it also introduces security risks.

An agent with unrestricted command execution could modify unrelated files, access sensitive information or perform operations outside the developer's intended scope.

Locdex separates model-generated actions from the permissions required to execute them.

The model may request permission to modify a file, but the runtime determines whether that operation is authorised.

The newer development architecture includes different permission modes, enabling developers to control whether an agent can read, modify or execute operations automatically.

For example, read-only repository inspection can proceed without approving every file read, while file modifications or more consequential actions may require explicit permission.

The system also includes workspace boundaries, restricted command execution, Git-operation controls and rollback mechanisms.

Sandboxing provides another layer of control by restricting the environment in which development commands execute. However, the actual isolation guarantees depend on the operating system and selected backend.

This distinction is particularly important because application-level permission checks do not provide the same protections as complete operating-system isolation.

One problem associated with coding agents is determining whether a generated solution actually works.

A language model may correctly identify a bug but introduce another problem while modifying the code. It may also report successful completion without adequately checking the resulting application.

Locdex treats verification as a separate part of agent execution.

After making modifications, the runtime can execute relevant tests, inspect results and use verification outcomes to determine whether further work is required.

The newer architecture also introduces change journals and checkpoints to support recovery when tasks fail.

This becomes especially important during operations involving multiple files or several execution steps.

We're also exploring user-defined multi-agent workflows, where different agents can perform related development tasks through isolated working environments.

However, additional autonomy increases the need for reliable permissions, verification and conflict management.

Our priority is to establish these foundations before expanding the level of automation.

Locdex has evolved considerably from our initial idea of a local coding assistant.

The broader objective is now to develop a reusable execution environment capable of supporting different coding models, hardware configurations and development workflows.

Much of our current work involves improving context efficiency, model routing, repository intelligence and execution reliability.

We also intend to develop benchmarking infrastructure to evaluate how different models perform across coding tasks, with particular attention to task completion, latency, hardware consumption and inference costs.

The project remains open-source under the MIT licence, with potential commercial extensions being explored for team workflows, enterprise governance and infrastructure services.

Local AI coding agents provide developers with greater flexibility in how artificial intelligence is integrated into software development. However, their effectiveness depends on more than the capabilities of the underlying language model.

Repository understanding, context construction, model selection, permission management and verification all contribute to the quality of an agent's execution.

Locdex is our attempt to bring these components together within an open-source, local-first development environment.

Although the newer architecture is still undergoing testing, we believe improving the infrastructure around language models is an important part of making AI coding agents more reliable and accessible.

Developers interested in local language models, coding agents or open-source development tools can explore the repository, test Locdex and contribute feedback.

GitHub: https://github.com/Locdex/Locdex

Development branch: https://github.com/Locdex/Locdex/tree/architecture-v3.2-ready