AI Agent Memory Needs Garbage Collection, Not Just Storage

TL;DR — Most long-term memory systems for AI agents only know how to write and append — never to forget, consolidate, or resolve contradictions. The result is memory that grows stale, noisy, and internally inconsistent, quietly degrading agent behavior over time. Agent memory needs an explicit lifecycle — decay, consolidation, and deletion — borrowed from database garbage collection, not just a bigger vector index.

Every long-term memory system for AI agents I've seen shares the same silent assumption: once something is written to memory, it stays there forever. New facts get appended. Old facts never get challenged, aged out, or reconciled. The memory store only grows. This feels safe — deleting information feels riskier than keeping it — but it's exactly backwards. An agent that remembers everything, forever, without a lifecycle policy, doesn't become smarter. It becomes unreliable in a way that's hard to debug because nothing looks broken. The retrieval call succeeds. The embeddings are fine. The agent is just quietly wrong, more often, as memory accumulates.

It's easy to see why append-only won. Writing memory is a tractable engineering problem: summarize a conversation, embed it, insert it into a vector store, done. Deciding when a memory is stale, contradicted, or no longer relevant is a much harder problem, so most systems skip it entirely. The write path gets built. The lifecycle does not.

This works fine in a demo. An agent with ten memories retrieves the right one almost by default — there's nothing else to compete with it. The failure mode only shows up under duration and volume, which is exactly the condition every production agent eventually hits. A support agent that's been running for a year has thousands of memories about a single customer: old addresses, canceled subscriptions, resolved complaints, preferences that have since changed twice. None of that was deleted. All of it is still searchable, still embeddable, still eligible to be retrieved into context on the next query.

Three specific failure modes show up, and they compound.

Contradiction without resolution. The user said they wanted email notifications in March and texted to disable them in October. Both memories are true statements about things that happened. Neither is marked as superseded. The agent has no way to know which one reflects current reality, so retrieval surfaces whichever is semantically closer to the query — which has nothing to do with which one is correct.

Retrieval dilution. Vector search finds nearest neighbors, not ground truth. As the memory store grows, the signal-to-noise ratio in any given retrieval drops. Ten near-duplicate memories about a resolved issue can crowd out the one memory that's actually relevant now, simply by weight of numbers. This is the same problem as index bloat in a database, except nobody's running a vacuum.

Staleness masquerading as personalization. The agent confidently uses a fact that was true once and presents it as current. This is worse than having no memory at all, because it's wrong with confidence. A system with no memory asks a clarifying question. A system with stale memory guesses and sounds certain doing it.

None of these show up in evaluation suites built around short-horizon interactions. They only appear after months of accumulated state — which is precisely why they're underrated. The bug is invisible until the memory store is old enough to have a past.

This is not a new problem in computing. It's the problem every long-running stateful system eventually faces, and the systems world has several working answers that the AI memory conversation has mostly ignored.

Database engines don't keep every row version forever — MVCC implementations garbage-collect old versions once nothing can observe them. Caches don't keep every entry — TTLs and LRU eviction exist precisely because unbounded retention degrades hit rate, not just disk usage. Even version control systems that preserve full history separate "history you can audit" from "state you actually operate on" — nobody runs their application off a full git log; they run it off a checked-out working tree that reflects current truth.

Agent memory needs the same separation. There's a difference between an audit log — what happened, immutable, timestamped, kept for provenance — and a working memory — a current, reconciled view of what's true right now, which is what should actually be injected into context. Most agent memory systems collapse these into one layer and retrieve directly from the audit log. That's the architectural mistake.

A memory system that behaves well over time needs at least four things the append-only version lacks.

Explicit supersession. When a new fact contradicts an old one, the write path should mark the old memory as superseded rather than leaving both live. This requires the write step to actually check for conflicts against existing memory before inserting — a retrieval-then-write pattern, not a blind append.

Confidence decay. Not all facts age the same way. A user's stated preference from a year ago should carry less retrieval weight than one from yesterday, even if both are technically still "true" in the log. Decaying confidence as a function of time, separate from semantic similarity, gives recency a lever that pure vector distance doesn't provide.

Consolidation. Ten similar memories about the same resolved issue should periodically collapse into one summarized memory. This is directly analogous to compaction in log-structured storage — merge, summarize, discard the redundant originals, keep the audit trail elsewhere if you need it for compliance.

Actual deletion, with a policy. Some memories should leave the retrievable store entirely — not be hidden behind a filter, but gone from the index the agent searches. This is uncomfortable for teams used to thinking of data retention as pure upside, but an agent's working memory is not a data warehouse. Its job is to be useful right now, not comprehensive.

The deeper issue is organizational, not technical. Memory lifecycle management doesn't fall neatly under any existing role. It's not quite data engineering, because it's about semantic truth, not schema. It's not quite ML, because there's no model being trained. It's not quite application logic, because the policy has to run continuously, in the background, independent of any single user request. Teams build the write path because it's someone's job to build storage. Almost nobody owns the decay curve, which is why it defaults to never happening.

If your agent is going to run for months rather than minutes, treat memory the way you'd treat any other long-running stateful system: write path, read path, and — the part everyone skips — an explicit policy for what gets demoted, merged, or deleted, and when. Storage was never the hard part. Knowing what to stop trusting is.