How I Self-Hosted a Mem0 Memory Stack (So My AI Agent Remembers)

Posted on Oct 20 • Originally published at blog.hoelee.com

Originally published on my blog — read it there with the full context, code and updates.

An AI agent that forgets everything between conversations is a tool you have to reintroduce yourself to every session. My AI assistant (the one that runs this very blog's workflow) used to start each chat from zero — until I gave it a memory layer: a self-hosted mem0 stack that stores facts it learns, retrieves them semantically, and survives across sessions.

This is the story of how I self-hosted it on my Synology NAS — private, fully under my control, no data leaving my LAN.

A memory layer changes what an agent can actually do for a business:

For anyone running AI agents (support bots, research assistants, personal automation) the question isn't should the agent remember — it's whether the memory is a private asset or a third party's.

Mem0 is "memory in a box" for LLM apps. Instead of you hand-rolling an embeddings table and a similarity search, it gives you a small API that does the three things memory needs:

The "memory" is the extracted fact. The "retrieval" is vector similarity. You get the agent even if you never touch a vector database yourself.

Three containers on my Synology DS1821+, as a Portainer compose stack:

┌──────────────┐ /search & /memories ┌──────────────────┐ │ AI agent / │ ──── X-Api-Key + JSON ───────▶ │ mem0 API:20015 │ │ Hermes / bot │ └────────┬─────────┘ └──────────────┘ │ pgvector ▼ PostgreSQL 17 + pgvector ┌──────────────┐ model calls (extract) ┌──────────────────┐ │ OpenAI etc. │ ◀───────── LiteLLM ──────────│ fact-classifier │ └──────────────┘ └──────────────────┘ Enter fullscreen mode Exit fullscreen mode A representative compose looks like this:

services: mem0-postgres: image: pgvector/pgvector:pg17 environment: POSTGRES_USER: mem0 POSTGRES_PASSWORD: ${MEM0_DB_PASSWORD} POSTGRES_DB: mem0 volumes: -./data/pg:/var/lib/postgresql/data restart: unless-stopped mem0-litellm: image: ghcr.io/berriai/litellm:main-stable command: [ " --config", " /app/config.yaml" ] volumes: -./litellm-config.yaml:/app/config.yaml restart: unless-stopped mem0-api: build:./mem0-api environment: OPENAI_API_KEY: ${OPENAI_API_KEY} OPENAI_API_BASE: http://mem0-litellm:4000/v1 # go through the gateway MEM0_API_KEY: ${MEM0_API_KEY} # X-Api-Key for /search & /memories POSTGRES_URL: postgresql://mem0:${MEM0_DB_PASSWORD}@mem0-postgres/mem0 ports: - " 20015:8000" restart: unless-stopped Enter fullscreen mode Exit fullscreen mode (Mine is built from the mem0 repo with the model/fact-extraction directed through LiteLLM; the exact image tag is me maintaining a local build, and I keep the real API key out of the compose file — it's an env secret.)

Once it's up, the API is refreshingly small. Add a fact (fast, verbatim):

curl -X POST http://192.168.1.1:20015/memories \ -H "Content-Type: application/json" \ -H "X-Api-Key: $MEM0_API_KEY " \ -d '{ "user_id": "customer-2211", "text": "Customer 2211 prefers email over WhatsApp for order updates." }' Enter fullscreen mode Exit fullscreen mode Retrieve the most relevant memories later:

curl -X POST http://192.168.1.1:20015/memories/search \ -H "Content-Type: application/json" \ -H "X-Api-Key: $MEM0_API_KEY " \ -d '{ "user_id": "customer-2211", "query": "how should I reach this customer?", "limit": 5 }' Enter fullscreen mode Exit fullscreen mode Then inject the returned facts into the agent's system prompt (or context) before it answers. That's the whole loop: store facts, retrieve what's relevant, use it in the next reply.

The route of my stack is /search and /memories without a /v1 prefix (an early wrapper that assumed /v1 404'd for a while), and auth is a plain X-Api-Key header. Small details, but they cost me time to discover.

A private, self-hosted memory layer answered by two small HTTP calls. My agent now remembers customers, preferences, and decisions across sessions — and because the stack sits in my own PostgreSQL behind my own network, none of that leaves my control.

If you run AI agents that keep forgetting their context, self-hosting mem0 is one of the highest-value, lowest-friction upgrades you can make — and you can do it without sending your memory to someone else's database.

I build and self-host AI agents, memory stacks, websites, and infrastructure for businesses. If you'd like your assistant to actually remember your customers — or you want to talk about hiring me — I'd love to help:

I write about self-hosted infrastructure, DevOps and the failures behind them at blog.hoelee.com.