Madeleine MCP

madeleine

Involuntary memory for AI agents. A place, an hour, a sound, a photo or a sensor reading brings back a related memory on its own, without the agent having to ask. It works the way the taste of a madeleine brought back Proust's childhood.

Local-first (SQLite) · MCP server + Python library · no telemetry

Most agent memory is voluntary: the agent decides to search, writes a query, and gets results. madeleine adds the other pathway. Every memory is stored with the sensory fingerprint present when it was encoded: where, when, what was said, seen or heard. Incoming cues from webhooks, phones, smart homes and wearables are matched against those fingerprints all the time. When enough channels line up, the memory is queued as an involuntary recollection, and the agent receives it with its next prompt. This is encoding specificity applied to agents (concepts).

60-second quickstart

# 1. See the whole loop, offline (nothing is downloaded, nothing leaves your machine)
uvx madeleine-mcp demo

The demo remembers three moments. Then a cue arrives that carries only a place and a time of day: 30 m from grandmother's house, on a Sunday afternoon. The matching memory surfaces by itself, is delivered through the MCP tool pending_recollections, and does not fire again for a repeated cue:

[madeleine 0.80 via geo+time] Sunday afternoons in grandmother's kitchen: lime-blossom tea and a madeleine (2019-04-14)
  channel similarity geo=0.97, time=1.00; salience x1.16, age decay x0.70

# 2. Give Claude Code the tools
claude mcp add madeleine -- uvx madeleine-mcp serve

3. Let involuntary recollections reach every prompt. Add this to .claude/settings.json:

{"hooks": {"UserPromptSubmit": [{"hooks": [
  {"type": "command", "command": "uvx madeleine-mcp hook claude-code", "timeout": 10}
]}]}}

The hook works even if your client ignores MCP notifications. See examples/claude_code for options such as --sense-prompt, which treats what you type as a cue too.

Other MCP clients

The server is a standard stdio MCP server (uvx madeleine-mcp serve), so any MCP client can use the five tools. Involuntary delivery on every prompt (step 3 above) is built for Claude Code; in other clients the agent receives recollections by calling pending_recollections, which the server's instructions ask it to do at the start of each turn.

{"mcpServers": {"madeleine": {"command": "uvx", "args": ["madeleine-mcp", "serve"]}}}

{"servers": {"madeleine": {"type": "stdio", "command": "uvx", "args": ["madeleine-mcp", "serve"]}}}

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {"madeleine": {"type": "local", "command": ["uvx", "madeleine-mcp", "serve"], "enabled": true}}
}

docker build -t madeleine-mcp https://github.com/fabianimv/madeleine.git
docker run -i --rm -v madeleine-data:/data madeleine-mcp        # stdio MCP server

Semantic text needs the [text] extra. The base install is small and offline. Without the extra, text is matched lexically by a hashing embedder, so "car" and "automobile" count as unrelated. For real sentence embeddings (this pulls in PyTorch):

claude mcp add madeleine -- uvx --from 'madeleine-mcp[text]' madeleine-mcp serve

Images ([image], open_clip) and sound ([audio], LAION CLAP) are opt-in the same way. [all] installs everything. Models download lazily on first use, never at import.

How it works

flowchart LR
    subgraph Sources
      HA[Home Assistant] -->|webhook| HTTP
      IOS[iOS Shortcuts] -->|webhook| HTTP
      AG[Agent] -->|MCP tools| MCP
    end
    HTTP[HTTP API /cues] --> ENG
    MCP[MCP server] --> ENG
    subgraph ENG[engine]
      direction TB
      CH["channels<br/>adapter → embedder → similarity"] --> FU["fusion<br/>base · coverage · salience · decay"]
      FU --> RF{"push?<br/>threshold + refractory"}
    end
    ENG <--> DB[(SQLite + sqlite-vec)]
    RF -->|enqueue| Q[(recollection queue)]
    Q -->|drain| HOOK[Claude Code hook]
    Q -->|drain / notify| MCP

• Channels are pluggable senses. Each one has an adapter (raw signal → normalised observation), an embedder (observation → vector) and a similarity function (→ [0, 1]). Built in: text, time (hour, weekday and season, all cyclical), geo (distance on the sphere), biometric, image, audio. smell is a documented stub for future e-noses.
• Recall combines per-channel similarity over the channels the cue and the memory share, adjusts for how much of the cue is covered, boosts salient memories, and decays with age down to a floor, so decades-old memories stay reachable. The formula is short and explainable; every result says which channels matched (ADR 0001).
• Two modes. Pull: the agent asks (recall). Push: a cue arrives (sense, POST /cues) and memories that clear the threshold are queued without anyone asking.
• Habituation. A memory that fired stays quiet for a refractory period (default 6 h). Walking past the same café all afternoon reminds you of it once.
• Context alone never fires. "It's 9 am" matches every memory made at 9 am, so time (and biometric) can strengthen a match but never cause one.

Sending cues

madeleine serve --transport http          # REST + MCP (streamable HTTP at /mcp/) on 127.0.0.1:8765
curl -s localhost:8765/cues -H 'content-type: application/json' \
  -d '{"lat": 48.30195, "lon": 1.24650, "signals": [{"channel": "text", "value": "smell of lime-blossom tea"}]}'

Ready-made integrations: Home Assistant (arriving home, media starts playing, wearable sensors), iOS Shortcuts ("Arrive at location" personal automation), curl.

MCP surface:

• Tools: remember, recall, sense, pending_recollections, forget.
• Resource: recollections://pending, with resources/updated notifications when something surfaces. These notifications also work across processes that share the database.

Privacy and security defaults

• Local-first. Everything lives in one SQLite file (~/.madeleine/madeleine.db, owner-only permissions). There is no telemetry and no cloud.
• Raw media is not kept. Image and audio bytes are embedded and then discarded. Only vectors and metadata are stored, unless you pass store_raw=True (per call or MADELEINE_STORE_RAW=1).
• No local file access by default. Clients must send media bytes. Allowing file paths takes madeleine serve --allow-file-uris.
• HTTP binds to 127.0.0.1. Host and Origin checks block DNS rebinding. Set MADELEINE_API_TOKEN to require a bearer token. Binding to another interface without a token is refused unless you pass --insecure-no-auth.
• forget deletes everywhere: memory, vectors, raw bytes, trigger history and queued recollections, with SQLite secure_delete on.
• Recollections are data, not instructions. The hook wraps them in delimiters and labels them as such. Anyone who can write memories can still put text in your agent's context, so keep write access to yourself. See SECURITY.md.

Benchmark (and where it fails)

madeleine bench runs a synthetic benchmark: 500 memories across 3 cities, 300 cues with known ground truth, distractors and hard negatives. It is reported for both text backends. Full tables, methodology and blind spots are in docs/benchmarks.md.

madeleine bench                                        # offline hash text backend
madeleine bench --text-backend sentence-transformers   # semantic, needs the [text] extra

By cue category (hit rate = how often the right memory surfaced on its own):

Latency with 500 memories, in pure Python: sense p50 11 ms / p95 23 ms with the hash backend, p50 32 ms / p95 63 ms with the semantic one (CPU).

Honest limitations

• Involuntary recall is tuned to be conservative: no negative cue fired, but a third (semantic) to half (hash) of positive cues still do not fire their memory.
• Both text backends are calibrated on the benchmark generator (seed 1; results are seed 0, and seeds 2 and 3 agree). Without that, the hash backend under-scored paraphrases and MiniLM over-scored unrelated sentences. The data is synthetic English text, so real-world numbers will differ.
• Altered media never matches in this benchmark, which uses the hash embedder for images and sound. That needs the CLIP and CLAP extras, whose similarity rescaling has only been calibrated on synthetic inputs.
• Place + hour cues saturate (3 fires per cue), and only 56 % of those fall in the strict "same place, same hour ±1 h" ground truth.
• The fusion is a hand-designed formula, not a learned one. There is no multi-user auth and no UI. The olfactory channel is a stub.
• Hour-of-day uses each timestamp's own UTC offset. Send local offsets (+02:00), or "evening" means UTC evening.

Configuration

CLI: madeleine serve | remember | sense | recall | pending | forget | demo | bench | hook claude-code (madeleine --help).

As a Python library

from madeleine.config import load_config
from madeleine.core.serde import context_from_fields
from madeleine.engine import create_engine

engine = create_engine(load_config(db_path="memories.db", text_backend="hash"))
now = engine.now()
engine.remember("Ana proposed at the Eiffel Tower",
                context=context_from_fields(now=now, lat=48.8584, lon=2.2945, salience=0.9))
cue = engine.build_cue(context=context_from_fields(now=now, lat=48.8585, lon=2.2946))
for rec in engine.sense(cue):          # involuntary: fires on place (+ time) alone
    print(rec.to_prompt_line())

Write your own sense with an Adapter, an Embedder and a similarity function (docs/channels.md). ImageBind is deliberately not a dependency because its license is CC-BY-NC, but you can plug it in yourself as an Embedder.

Development

See CONTRIBUTING.md. Run uv sync once, then before every commit: uv run pytest, uv run ruff check src tests, uv run mypy. The architecture is in ARCHITECTURE.md and the design decisions are in docs/adr.

License

Apache-2.0. See LICENSE.