Can a local model replace Jev in my Claude Code memory tool? I tested two

Someone on GitHub asked if jevmem could run fully local.

Fair question. Today jevmem needs a paid TypeSafe API key: it sends each message, with common secrets removed, to TypeSafe's Jev API to decide what's worth saving.

Ollaya runs open decision models on your own machine, and it answers in the same format as Jev's API. So I pointed jevmem at it.

It connected with no code change: one URL, any key, and the model's name in the config.

Then I ran the same tests I use for Jev.

2 Oct 2026. M4 Mac with 16 GB, Ollaya 0.9.0. One run each, next to a Jev run the same afternoon.

Deciding what to save (66 held-out turns)

Both local models needed the client timeout raised from 10 s to 180 s.

On a laptop, not yet, so jevmem 0.7.3 still uses Jev. The accurate model is over 100× slower. The fast one is wrong too often.

But 60 of 66 is close. And on JevBench, larger open models score near Jev when they run on servers. So this may change with better hardware, or the next models.

If you have an NVIDIA GPU and want to try, scripts/local-model-ollaya.sh in the repo runs each test against a local server. I'd love to see your numbers.

Repo: https://github.com/Avinash-jetwani/jevmem · Results: https://avinash-jetwani.github.io/jevmem/results/#can-a-local-model-do-jevs-job · JevBench: benchmarkheaven.com/jev-models

Written with help from Claude. The numbers are from my own test sets, run once each, and the results files are in the repo.