# Your AI Assistant Remembered the Fact. It Still Got the Answer Wrong

An assistant remembers that Sarah works at Acme.

The assistant confidently answers with her new employer.

It retrieved a real fact. It identified the right person. It may even have selected the most recently updated record.

And it still answered the wrong question.

This is a useful way to think about AI memory failures: sometimes the problem is not forgetting. It is remembering something without preserving the conditions under which it is relevant.

For an engineering design, I would treat validity as a first-class concern—not metadata added after retrieval.

There are at least two distinct times to represent:

These timestamps answer different questions.

“Where did I live in April?” asks about the world.

“What did you know about my location in April?” asks about the system’s knowledge at that point.

Treating both as a single message timestamp loses that distinction.

This is the motivation behind a bitemporal representation: separate valid time—when a fact held in the world—from transaction or ingestion time—when the system recorded it. Graphiti’s architecture explicitly distinguishes these timelines and tracks validity and record lifecycle timestamps. arxiv

This is not a claim that every assistant needs a temporal graph database. It is a claim that the data model needs to distinguish questions the product promises to answer.

Imagine these statements arriving over time:

A memory pipeline that simply overwrites current_employer risks collapsing several different concepts:

Do not turn every newly extracted sentence into an immediately valid replacement fact.

A plan is not an event. A correction is not always a new state. A later message can change how an earlier message should be interpreted.

Temporal memory systems such as Graphiti preserve superseded facts through invalidation rather than simply deleting them, allowing current and historical relationships to remain distinguishable. getzep

But preserving history is only part of the design. The reader must also choose the right historical state for the question.

I would separate three layers in a memory implementation.

Keep the source message, speaker, timestamp, and enough surrounding context to interpret it.

What does the system think the source claims?

{ "subject_id": "person_sarah_01", "predicate": "employed_by", "object_id": "company_acme_01", "assertion_type": "reported_state", "recorded_at": "2026-06-10", "valid_from": null, "valid_to": null, "source_message_id": "message_184" } Enter fullscreen mode Exit fullscreen mode This is an illustrative schema, not a complete temporal database design.

The null dates matter. If the source does not establish when employment started, the extractor should not invent a start date.

What should the assistant currently believe?

A current-state view can select the assertions relevant to “Where does Sarah work now?”

A historical view can select evidence relevant to “Where did Sarah work when we met?”

These views should point back to the underlying evidence.

The architectural goal is to make interpretations revisable without losing the information used to produce them.

One is your colleague. The other is your sister.

A name alone cannot distinguish them in this example. But “name plus relationship” is not necessarily a permanent identifier either: colleagues become friends, managers change, and relationships can overlap.

I would use stable internal entity IDs, with names and relationships stored as evidence-backed attributes.

For ambiguous references, the write path should have more than two options.

Create entity or Merge into existing entity Enter fullscreen mode Exit fullscreen mode allow:

Create entity Link with sufficient evidence Hold as unresolved Ask for clarification Enter fullscreen mode Exit fullscreen mode A system that can preserve uncertainty does not have to turn every ambiguous mention into a confident merge.

They concern the same person and topic, but they ask for different evidence.

For this design, I would represent query intent explicitly enough to distinguish:

The retrieval pipeline could then look like this:

Question ↓ Entity resolution ↓ Temporal and question-intent interpretation ↓ Candidate retrieval ↓ Validity and provenance checks ↓ Evidence selection ↓ Answer or abstention Enter fullscreen mode Exit fullscreen mode This is a proposed decomposition, not an argument that every box requires a separate model call.

The important part is that “most similar” and “appropriate evidence” are not treated as interchangeable.

Suppose a fact concerns both Sarah and Acme.

It should be discoverable through either entity.

For a structured implementation, I would prefer one canonical fact record referenced by both entity indexes rather than two independently editable copies:

Sarah index ──┐ ├── fact_271 Acme index ───┘ Enter fullscreen mode Exit fullscreen mode That gives corrections one authoritative target.

Materialized copies may still be useful for performance, but then their consistency needs to be an explicit engineering responsibility.

A chapter metaphor can make navigation intuitive. It should not accidentally dictate an update model that is difficult to maintain.

Imagine a memory system with three stages:

Conversation → Extraction → Filing → Retrieval Enter fullscreen mode Exit fullscreen mode A wrong answer could originate anywhere:

Improving the retriever will not reveal which of these happened.

Before choosing the next model to train, I would run an ablation:

Condition A is a reference condition for retrieval with verified inputs—not a theoretical upper bound.

The other conditions help investigate which upstream errors are responsible for the observed failures.

The evaluation should report those failures separately, rather than reducing every error to “memory recall.”

A useful test set should contain more than questions with known, neatly stored answers.

For each case, define acceptable behavior in advance.

“I know Sarah changed employers, but I don’t have enough information to determine where she worked on that date.”

An evaluation that always requires a factual answer cannot tell you whether the system handles missing evidence appropriately.

Retrieval evaluation is useful, but it does not automatically establish the performance of the complete assistant.

For a proposed system, I would track at least three groups of measurements.

A selective reader may reduce context size, but that remains an empirical question until the full reading loop is integrated and measured.

Likewise, an extra retrieval step might save prompt tokens while adding latency. The relevant trade-off belongs in the end-to-end results, not only in an architectural diagram.

Semantic retrieval remains useful. It can help locate evidence when the user’s wording differs from the stored text.

But my design recommendation is to place it inside a broader lifecycle:

Capture → Interpret → Resolve identity → Record provenance → Establish validity → Retrieve → Verify → Correct Enter fullscreen mode Exit fullscreen mode Existing systems already address parts of this lifecycle. Graphiti documents temporal invalidation and historical relationships; Mem0 exposes a memory-history interface for tracking how an individual memory has changed. Neither should be reduced to “just a pile of embeddings.” help.getzep

The useful engineering question is therefore not simply:

“What evidence does this assistant preserve, how does it revise that evidence, and under what conditions is it allowed to use it?”

An assistant that remembers yesterday’s fact is useful.

An assistant that can explain whether the fact is still true—and recognize when it cannot tell—is a more demanding system to build.