Building CedroDB: Exploring Local-First Storage for AI Agents with Rust

Modern AI applications are becoming increasingly complex.

An AI agent may need to remember previous interactions, retrieve relevant information, maintain its execution state, and recover interrupted workflows.

These capabilities often require combining databases, search engines, job queues, and persistence mechanisms.

But what if we could explore bringing several of these capabilities into a single embedded storage engine?

CedroDB is an experimental, open-source, local-first database engine developed in Rust, with native libraries for Python and Node.js/TypeScript.

It explores combining document storage, search, durable workflows, and persistent agent state without requiring a separate database server.

The goal isn't to replace established databases. It's to experiment with a storage architecture designed around the needs of local applications and AI agents.

Imagine building an AI assistant that needs to:

Implementing these capabilities usually involves coordinating multiple components.

CedroDB explores an alternative: managing them through a unified local storage engine.

One particularly interesting capability is storing an agent checkpoint together with related document and outbox writes in a single atomic transaction.

This can help avoid inconsistent local state when a workflow is interrupted.

Store structured JSON documents and perform atomic operations across collections.

This allows applications to retrieve local information using different search strategies.

The application remains responsible for generating embeddings.

Agent checkpoints allow applications to preserve execution state.

CedroDB also supports revision checks to prevent stale checkpoint updates.

Persistent job queues support retries, leases, and recoverable workflows.

Applications remain responsible for running workers and handling external operations.

CedroDB includes durable changefeeds, backup capabilities, and integrity verification mechanisms.

These features help support local integrations and recovery workflows.

Rust provides several characteristics that make it interesting for building storage engines:

CedroDB uses Rust for its core engine while exposing native bindings for other programming languages.

pip install cedrodb Enter fullscreen mode Exit fullscreen mode Example:

from cedrodb import Database with Database ( "./app.cedro " ) as db: notes = db. collection ( " notes " ) notes. put_many ({ " note-1 ": { " state ": " pending " }, " note-2 ": { " state ": " completed " } }) notes. create_index ( " state " ) results = notes. find_page ([ { " field ": " state ", " op ": " eq ", " value ": " pending " } ]) print ( results [ " records " ]) Enter fullscreen mode Exit fullscreen mode No external database server is required.

npm install cedrodb Enter fullscreen mode Exit fullscreen mode The native Node.js/TypeScript bindings provide access to the Rust storage engine.

You can explore the API and examples directly in the GitHub repository.

CedroDB is currently an experimental project (v0.9.0).

These trade-offs are part of the project's ongoing exploration.

SQLite and other mature databases remain better choices for many production workloads.

The roadmap toward CedroDB 1.0 focuses on validation, reliability, performance testing, and improvements to the storage engine.

Community feedback is particularly valuable in identifying architectural limitations and practical use cases.

Official Website: https://www.ltavarez.me/cedrodb

GitHub Repository: https://github.com/uppy19d0/cedrodb

npm Package: https://www.npmjs.com/package/cedrodb

Python Package: https://pypi.org/project/cedrodb/

Building an embedded database is an interesting engineering challenge, especially when combining storage, retrieval, and recoverable workflows.

Would you use a single embedded storage engine for AI agent memory, document search, and durable jobs, or would you prefer separate specialized components?

Feedback, technical discussions, and open-source contributions are welcome.

If you find CedroDB interesting, consider giving the repository a ⭐ on GitHub.

Built with Rust. Accessible from Python and Node.js. Open source.