madeleine

madeleine

Involuntary memory for AI agents. A place, an hour, a sound, a photo or a sensor readingbrings back a related memory on its own, without the agent having to ask. It works the waythe taste of a madeleine brought back Proust's childhood.

Local-first (SQLite) · MCP server + Python library · no telemetry

Most agent memory is voluntary: the agent decides to search, writes a query, and getsresults. madeleine adds the other pathway. Every memory is stored with the sensoryfingerprint present when it was encoded: where, when, what was said, seen or heard.Incoming cues from webhooks, phones, smart homes and wearables are matched against thosefingerprints all the time. When enough channels line up, the memory is queued as aninvoluntary recollection, and the agent receives it with its next prompt. This isencoding specificity applied to agents (concepts).

60-second quickstart

# 1. See the whole loop, offline (nothing is downloaded, nothing leaves your machine)
uvx madeleine-mcp demo

The demo remembers three moments. Then a cue arrives that carries only a place and a time ofday: 30 m from grandmother's house, on a Sunday afternoon. The matching memory surfaces byitself, is delivered through the MCP tool pending_recollections, and does not fire again fora repeated cue:

[madeleine 0.80 via geo+time] Sunday afternoons in grandmother's kitchen: lime-blossom tea and a madeleine (2019-04-14)
  channel similarity geo=0.97, time=1.00; salience x1.16, age decay x0.70

# 2. Give Claude Code the tools
claude mcp add madeleine -- uvx madeleine-mcp serve

3. Let involuntary recollections reach every prompt. Add this to .claude/settings.json:

{"hooks": {"UserPromptSubmit": [{"hooks": [
  {"type": "command", "command": "uvx madeleine-mcp hook claude-code", "timeout": 10}
]}]}}

The hook works even if your client ignores MCP notifications. Seeexamples/claude_code for options such as --sense-prompt,which treats what you type as a cue too.

Other MCP clients

The server is a standard stdio MCP server (uvx madeleine-mcp serve), so any MCP client canuse the seven tools. Involuntary delivery on every prompt (step 3 above) is built forClaude Code; in other clients the agent receives recollections by callingpending_recollections, which the server's instructions ask it to do at the start of eachturn.

Claude Desktop (claude_desktop_config.json) and Cursor (~/.cursor/mcp.json):

{"mcpServers": {"madeleine": {"command": "uvx", "args": ["madeleine-mcp", "serve"]}}}

VS Code (.vscode/mcp.json):

{"servers": {"madeleine": {"type": "stdio", "command": "uvx", "args": ["madeleine-mcp", "serve"]}}}

OpenCode (opencode.json):

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {"madeleine": {"type": "local", "command": ["uvx", "madeleine-mcp", "serve"], "enabled": true}}
}

Docker (memories persist in the madeleine-data volume):

docker build -t madeleine-mcp https://github.com/fabianimv/madeleine.git
docker run -i --rm -v madeleine-data:/data madeleine-mcp        # stdio MCP server

Semantic text needs the [text] extra. The base install is small and offline. Withoutthe extra, text is matched lexically by a hashing embedder, so "car" and "automobile"count as unrelated. For real sentence embeddings (this pulls in PyTorch):

claude mcp add madeleine -- uvx --from 'madeleine-mcp[text]' madeleine-mcp serve

Images ([image], open_clip) and sound ([audio], LAION CLAP) are opt-in the same way.[all] installs everything. Models download lazily on first use, never at import.

How it works

flowchart LR
    subgraph Sources
      HA[Home Assistant] -->|webhook| HTTP
      IOS[iOS Shortcuts] -->|webhook| HTTP
      AG[Agent] -->|MCP tools| MCP
    end
    HTTP[HTTP API /cues] --> ENG
    MCP[MCP server] --> ENG
    subgraph ENG[engine]
      direction TB
      CH["channels<br/>adapter → embedder → similarity"] --> FU["fusion<br/>base · coverage · salience · decay"]
      FU --> RF{"push?<br/>threshold + refractory"}
    end
    ENG <--> DB[(SQLite + sqlite-vec)]
    RF -->|enqueue| Q[(recollection queue)]
    Q -->|drain| HOOK[Claude Code hook]
    Q -->|drain / notify| MCP

• Channels are pluggable senses. Each one has an adapter (raw signal → normalisedobservation), an embedder (observation → vector) and a similarity function (→ [0, 1]).Built in: text, time (hour, weekday and season, all cyclical), geo (distance on thesphere), biometric, image, audio. smell is a documented stub for future e-noses.
• Recall combines per-channel similarity over the channels the cue and the memory share,adjusts for how much of the cue is covered, boosts salient memories, and decays with agedown to a floor, so decades-old memories stay reachable. The formula is short andexplainable; every result says which channels matched(ADR 0001).
• Two modes. Pull: the agent asks (recall). Push: a cue arrives (sense, POST /cues)and memories that clear the threshold are queued without anyone asking.
• Habituation. A memory that fired stays quiet for a refractory period (default 6 h).Walking past the same café all afternoon reminds you of it once.
• Context alone never fires. "It's 9 am" matches every memory made at 9 am, so time(and biometric) can strengthen a match but never cause one.

Sending cues

madeleine serve --transport http          # REST + MCP (streamable HTTP at /mcp/) on 127.0.0.1:8765
curl -s localhost:8765/cues -H 'content-type: application/json' \
  -d '{"lat": 48.30195, "lon": 1.24650, "signals": [{"channel": "text", "value": "smell of lime-blossom tea"}]}'

Ready-made integrations: Home Assistant (arriving home,media starts playing, wearable sensors), iOS Shortcuts("Arrive at location" personal automation), curl.

MCP surface:

• Tools: remember, recall, sense, pending_recollections, get_memory, update_memory,forget.
• Resource: recollections://pending, with resources/updated notifications when somethingsurfaces. These notifications also work across processes that share the database.

Privacy and security defaults

• Local-first. Everything lives in one SQLite file (~/.madeleine/madeleine.db, owner-onlypermissions). There is no telemetry and no cloud.
• Raw media is not kept. Image and audio bytes are embedded and then discarded. Onlyvectors and metadata are stored, unless you pass store_raw=True (per call orMADELEINE_STORE_RAW=1).
• No local file access by default. Clients must send media bytes. Allowing file pathstakes madeleine serve --allow-file-uris.
• HTTP binds to 127.0.0.1. Host and Origin checks block DNS rebinding. SetMADELEINE_API_TOKEN to require a bearer token. Binding to another interface without atoken is refused unless you pass --insecure-no-auth.
• forget deletes everywhere: memory, vectors, raw bytes, trigger history and queuedrecollections, with SQLite secure_delete on.
• Recollections are data, not instructions. The hook wraps them in delimiters and labelsthem as such. Anyone who can write memories can still put text in your agent's context, sokeep write access to yourself. See SECURITY.md.

Benchmark (and where it fails)

madeleine bench runs a synthetic benchmark: 500 memories across 3 cities, 300 cues withknown ground truth, distractors and hard negatives. It is reported for both text backends.Full tables, methodology and blind spots are in docs/benchmarks.md.

madeleine bench                                        # offline hash text backend
madeleine bench --text-backend sentence-transformers   # semantic, needs the [text] extra

By cue category (hit rate = how often the right memory surfaced on its own):

Latency with 500 memories, in pure Python: sense p50 11 ms / p95 23 ms with the hashbackend, p50 32 ms / p95 63 ms with the semantic one (CPU).

Honest limitations

• Involuntary recall is tuned to be conservative: no negative cue fired, but a third (semantic)to half (hash) of positive cues still do not fire their memory.
• Both text backends are calibrated on the benchmark generator (seed 1; results are seed 0,and seeds 2 and 3 agree). Without that, the hash backend under-scored paraphrases and MiniLMover-scored unrelated sentences. The data is synthetic English text, so real-world numberswill differ.
• Altered media never matches in this benchmark, which uses the hash embedder for images andsound. That needs the CLIP and CLAP extras, whose similarity rescaling has only beencalibrated on synthetic inputs.
• Place + hour cues saturate (3 fires per cue), and only 56 % of those fall in the strict"same place, same hour ±1 h" ground truth.
• The fusion is a hand-designed formula, not a learned one. There is no multi-user auth and noUI. The olfactory channel is a stub.
• Hour-of-day uses each timestamp's own UTC offset. Send local offsets (+02:00), or "evening"means UTC evening.

Configuration

CLI: madeleine serve | remember | sense | recall | pending | show | edit | forget | demo | bench | hook claude-code(madeleine --help).

As a Python library

from madeleine.config import load_config
from madeleine.core.serde import context_from_fields
from madeleine.engine import create_engine

engine = create_engine(load_config(db_path="memories.db", text_backend="hash"))
now = engine.now()
engine.remember("Ana proposed at the Eiffel Tower",
                context=context_from_fields(now=now, lat=48.8584, lon=2.2945, salience=0.9))
cue = engine.build_cue(context=context_from_fields(now=now, lat=48.8585, lon=2.2946))
for rec in engine.sense(cue):          # involuntary: fires on place (+ time) alone
    print(rec.to_prompt_line())

Write your own sense with an Adapter, an Embedder and a similarity function(docs/channels.md). ImageBind is deliberately not a dependencybecause its license is CC-BY-NC, but you can plug it in yourself as an Embedder.

Development

See CONTRIBUTING.md. Run uv sync once, then before every commit:uv run pytest, uv run ruff check src tests, uv run mypy. The architecture is inARCHITECTURE.md and the design decisions are in docs/adr.

License

Apache-2.0. See LICENSE.