Self-hosted memory for AI agents: the complete guide

Self-hosted agent memory means your AI agent's long-term memory — every fact it retains about users, projects and decisions — lives in a database you run, typically PostgreSQL, instead of a third-party memory API. This guide covers why teams self-host, what "self-hosted" actually has to include to mean anything, and a working setup you can run in about ten minutes with Docker Compose.

Last updated: August 2026.

Why teams self-host agent memory

Four reasons come up over and over, and only one of them is about money.

Everything flows through memory. A memory layer is not a normal SaaS dependency. To be useful it has to see every conversation, every preference, every correction — the most sensitive stream your product produces. With a hosted memory API, that stream leaves your infrastructure on every request. For anyone handling customer data under GDPR, HIPAA, or a bank's vendor policy, that single fact ends the evaluation before pricing comes up.

Deletion has to be real. "Right to erasure" means erasure from the facts, the embeddings, the source events, and everything derived from them — with proof it happened. That's difficult to verify through someone else's API and straightforward to verify in your own database.

Vendor risk is not hypothetical in this category. Zep, one of the best-known memory products, discontinued its self-hostable Community Edition — the repository now carries a notice that it is no longer the product. Teams that built on it got a migration project they didn't plan. When memory is a hosted service, your agent's entire accumulated knowledge sits behind someone else's roadmap.

Latency. A memory lookup sits in the critical path of every agent response. A round-trip to an external API adds its own network time before your LLM call even starts. On your own hardware, retrieval can stay under 50 ms — numbers below.

What "self-hosted" has to include (read this before trusting the label)

The label gets stretched. A setup where the database is yours but every message is shipped to a hosted LLM for fact extraction is half self-hosted at best. Break any memory system into four parts and ask where each one runs:

One honest note, because it applies to every memory system including ours: anything that extracts structured facts from conversation uses an LLM somewhere. The meaningful differences are whether that call is in the request path (it shouldn't be — extraction can run async), and whether you can point it at your own endpoint, a local model, or a provider you've approved.

A working setup in ~10 minutes

This uses Haki, the memory layer we build — Apache 2.0, one docker compose up, no account anywhere in the flow. The same checklist logic applies to whatever you choose.

Prerequisites: Docker and uv.

git clone https://github.com/GetHaki/Haki && cd Haki

# PostgreSQL 16 + pgvector, Redis 7
docker compose up -d

# Dependencies (uv installs Python 3.12 if needed)
uv sync

# Database migrations
uv run alembic upgrade head

# API
uv run uvicorn app.main:app --port 8100

Then, in a second terminal, the part that matters — proving the memory loop actually works, not taking it on faith:

uv run haki connect --api-url http://localhost:8100
uv run haki verify

haki verify runs a complete scenario in a few seconds: a preference, then a change of mind in the same conversation, then a new conversation that queries memory. It has to serve the current value, keep the old one as superseded rather than deleting it, and attach the whole thing to a trace:

haki verify — subject usr_verify_91d952a5e06f

  ✔ capture     "Je préfère recevoir mes factures en français."    thr_35bb7ecf
  ✔ consolidate 1 fact(s) extracted                                0.2s
  ✔ capture     "En fait, envoie-les moi en anglais plutôt..."     thr_35bb7ecf (same thread)
  ✔ consolidate 1 supersession                                     0.1s
  ✔ context     NEW thread thr_3a21ef34                            0.0s

    recalled  invoice_language = {"language": "en"}   valid since 2026-08-11
    hidden    invoice_language = {"language": "fr"}   superseded
    trace     7c99a8de-4905-43b4-94df-21fb66492b3b

OK — your agent remembered across conversations, and it can prove it.  0.5s

The command exits non-zero if the stale value is still served, or if the old value isn't found as superseded. Serving the right answer by accident, with no link between the two facts, is not a memory that updates — it's a memory that got lucky.

The latency you should expect

Embeddings here are computed locally (ONNX on CPU, a 384-dimension multilingual model), so there are zero network calls in the retrieval path. Measured with the reproducible benchmark in the repo (scripts/benchmark_context.py, 100 queries per size):

Retrieval combines a pgvector HNSW index with Postgres full-text search, then scores only the top candidates — which is why the numbers don't degrade as memory grows.

What to demand from any self-hosted memory (the portable checklist)

Whether you pick Haki, assemble something on Graphiti, or build your own on pgvector, these six behaviors separate a memory from a cache. Test each one before trusting the system with production data:

1. Supersession, not overwrites. When a fact changes, the old value should be marked replaced and never served as current — but never silently deleted either. An UPDATE that destroys history will cost you the first time you have to debug why the agent said something.
2. Contradictions handled explicitly. Two conflicting values should be flagged and dated, not served at random.
3. Deletion that propagates. One forget call should remove the fact, its embeddings, its source events, and everything derived — and give you a receipt.
4. A trace per recall. "Why did the agent use this information?" should have an answer you can pull up in under a minute.
5. Isolation enforced by the database. If tenant separation only exists in application code, one missed filter is a data leak. Postgres row-level security makes the database itself refuse.
6. Idempotent writes. A network retry must never create a duplicate memory.

If a system can't demonstrate these, the self-hosting is cosmetic: you'll own the servers and none of the guarantees.

FAQ

Can I use my existing Postgres? Yes — the requirements are Postgres 16 with the pgvector extension. The compose file ships one, but nothing stops you from pointing at a database you already run.

Does it need an OpenAI key? Not for retrieval — embeddings are local and the hot path makes no network calls. Fact extraction runs async through a configurable provider (HAKI_LLM_PROVIDER), which can be any OpenAI-compatible endpoint, including one you host.

How is this different from just using pgvector? pgvector stores and searches vectors. It has no opinion about what happens when a fact changes, contradicts another, or must be erased. A memory layer is the lifecycle around the vectors — supersession, conflicts, deletion, traces. That's the part you'd otherwise build yourself, and the part this guide's checklist tests.

What about GDPR erasure? POST /v1/forget cascades through facts, embeddings, events and traces, and writes a timestamped receipt to a forget_receipts table. That receipt is your audit answer.