SuperBased Observer

SuperBased Observer

The exact tokens your AI provider billed you — cache splits,reasoning tokens, long-context surcharges — reconciled across 26coding tools, entirely on your own machine. Nothing you build hereever leaves your machine unless you opt a node into it.

Table of contents

• What it is in 30 seconds
• Install
• First-run walkthrough
• Dashboard tour
• MCP server — 25 cross-tool intelligence calls
• API proxy — accurate token capture + compression
• Architecture
• Teams & Org Visibility
• Security & control layer (guard)
• CLI reference
• Configuration
• Post-upgrade hygiene + recovery
• Build from source
• Contributing
• License

What it is in 30 seconds

A single local Go binary built on three things a hosted usage consolecan't give you, in order of how much they matter:

1. Proxy-accurate, cross-vendor cost attribution. An optional APIreverse proxy reads the token counts your provider actuallybilled — net input, 5m/1h cache read/write splits, reasoningtokens, long-context repricing — the same math your invoice uses,not a JSONL-derived estimate. It's accurate enough that it caughtits own bug: a Codex reasoning-token double-billing regressionObserver found and back-corrected months of history for(migration 058, shipped v1.18.0) — the kind of self-audit a vendorconsole has no incentive to run against itself.
2. Local-first, by construction. The watcher, proxy, dashboard,MCP server, and CLI make zero outbound calls on your behalf — notelemetry, no analytics, no remote reporting. An optional teamrollup server ships only hashed metadata by default; raw contentships only when the node opts in, never a remote admin toggle.Full details: PRIVACY.md.
3. One capture layer, every tool you actually use. 26 adapters —Claude Code, Codex, Cursor, Cline + Cline CLI, GitHub Copilot +Copilot CLI, Gemini CLI, OpenCode, Google Antigravity, Cowork,Hermes Agent, Kilo Code, Aider, Goose, Devin, Qoder, Crush, Grok,Kiro CLI, Kimi Code, Qwen Code, OpenClaw, Pi, and more — parsedinto one normalized schema, queryable from a local dashboard, anMCP server (so the tools themselves can query it), and a CLI.(Five of the 26 are *-web adapters for ChatGPT/Claude.ai/Gemini/Copilot/Perplexity in the browser — those need the browser-captureextension, which today only installs unpacked; every tool listedabove works out of the box.)

Raw token counting across tools is table stakes here — it's thesubstrate the accurate-cost layer above is built on, not the pitch.

Two planes, one binary. Plane B is coding-agent observability— desktop-first: capture, proxy-accurate cost, compression, cachetracking, and session handoff for every AI coding tool on yourmachine, with an optional team rollup server for org-wide spend.Plane A is general LLM-app observability — admin-server-first:OTLP trace/span capture, evals, and an LLM-as-judge input-admissionguardrail for an application you host, whose end users routethrough Observer. Most solo developers only ever touch Plane B; teamsadd Plane A when they're also running an LLM-powered product. Fullexplainer: superbased.app/docs/getting-started/two-planes(or docs/deployment-models.md if you're reading this in the repo).

It answers questions like:

• Where did this week's $147 Claude bill come from — which projects,models, sessions, tool calls? And is that number the same one myprovider's invoice would show?
• Did I spend more on Opus or Sonnet? Are my Sonnet sessions hittingthe long-context tier and getting repriced at 2×?
• How much did I waste re-reading files that hadn't changed sincethe last read in the same session?
• Could that trivial Opus session have been done by Sonnet for 1/5the cost?
• Across Claude Code, Cursor, and Codex working in the same repo,what files are touched by all three? Where are they stepping oneach other?
• What will my next message roughly cost — and how much of my 5-hourand weekly subscription limit is left before I hit it?
• Where did my own OpenTelemetry-instrumented agent spend its tokens— with the proxy's exact per-span cost where it routed through theproxy?

Install

Pick whichever package manager fits your environment — npm and PyPIship the same prebuilt binary from the same v* tag, versionnumbers kept in lock-step.

Via VS Code (Marketplace or Open VSX)

code --install-extension superbased.superbased-observer

The VS Code extension bundles the observer binary, lifts thedashboard / sidebar / status bar / file decorations into the editor,and contributes a terminal profile that pre-exports the proxy envvars so AI CLIs launched from it route through observerautomatically. Cursor, VSCodium, and Windsurf install the same VSIXvia Open VSX.

After install, VS Code's Get Started page surfaces an in-editorwalkthrough; the long-form user guide lives atdocs/vscode-extension-user-guide.mdand the command + settings reference is atdocs/vscode-extension.md.

Via npm (recommended for Node users)

npm install -g @superbased/observer
observer --version

Via pip / uv / pipx (recommended for Python users)

pip install superbased-observer            # plain pip
uv tool install superbased-observer        # uv (isolated env, fastest)
pipx install superbased-observer           # pipx (isolated env)
observer --version

Wheels ship for manylinux2014_{x86_64,aarch64},macosx_*_{x86_64,arm64}, and win_amd64. uv tool and pipxkeep the install isolated from your project's Python env — generallywhat you want for a CLI tool.

Via go install (latest main, builds locally)

go install github.com/marmutapp/superbased-observer/cmd/observer@latest
observer --version

Via direct download (pre-built per-platform archive)

Each tagged release attaches per-platform archives to theReleases page,verifiable against the published SHA256SUMS:

# Linux x64 example — substitute your platform + version.
VERSION=v1.6.21
PLAT=linux-x64
curl -L -O https://github.com/marmutapp/superbased-observer/releases/download/$VERSION/observer-$VERSION-$PLAT.tar.gz
curl -L -O https://github.com/marmutapp/superbased-observer/releases/download/$VERSION/SHA256SUMS
shasum -a 256 -c SHA256SUMS --ignore-missing
tar -xzf observer-$VERSION-$PLAT.tar.gz
./observer --version

The binary is pure Go — no CGO, no external runtime dependencies.SQLite storage is pure-Go via modernc.org/sqlite. Single staticbinary; scp it anywhere it runs. Same artifacts ship to npm and tothe Releases page (build-once-ship-everywhere CI), so the npm anddirect-download paths produce byte-identical binaries.

First-run walkthrough

# 1. Start everything: proxy + watcher + dashboard in one foreground
#    process (ctrl-c to stop). Hooks auto-register for every detected
#    AI tool, and the dashboard opens in your browser
#    (http://localhost:8081; suppress with --no-open).
observer start

# 2. (another shell) Backfill from existing session logs so the
#    dashboard has history immediately rather than starting empty.
observer scan

From here the dashboard drives. On an empty database the Overview tableads with a three-step onboarding checklist — and a demo modeoffer if you'd rather look around first: one click seeds a temporarysynthetic dataset so every chart renders with realistic data (yourreal observer.db is never read or written; a persistent banner marksdemo state and one click clears it). The two checklist steps thatmatter:

3. Route your AI tool through the proxy — accurate token countsand conversation compression both need it. On the Compressiontab's Proxy banner, click your tool's status pill, thenRoute through the observer proxy…: the button previews theexact file change (Claude Code: an env.ANTHROPIC_BASE_URL entryin ~/.claude/settings.json; Codex: an observer model providerin ~/.codex/config.toml) and writes only on confirm. Durable —every later session routes automatically. The same section of theSettings → Connected tools panel offers a per-tool setup wizard(hooks / MCP / routing, one consent click per write) and a Launchbutton. Prefer the terminal? The same routing ships as observer init, as session-scoped wrappers (observer claude / observer codex — no config writes), or as a plainexport ANTHROPIC_BASE_URL=http://localhost:8820 /OPENAI_BASE_URL=http://localhost:8820/v1.
4. Use your AI tool as normal. The checklist tracks the firstcaptured session; cost, compression, and cache numbers populatewithin minutes of real activity.

Optional — MCP registration. observer init additionally writesMCP server entries (and hook entries) into each AI tool's own configfiles (~/.claude/settings.json, ~/.claude.json,~/.cursor/mcp.json, ~/.codex/config.toml, …). Hooks default ON,MCP defaults ON; opt out per-side with --skip-hooks / --skip-mcp.Idempotent. observer start alone never registers the MCP server —MCP wiring is explicit-only, because it costs ~1,800 schema tokensper AI-client turn.

If you route Claude Code while MCP servers are registered, setENABLE_TOOL_SEARCH=true in the same environment. Claude Code'sSDK disables ToolSearch:optimistic deferred MCP loading wheneverANTHROPIC_BASE_URL is set, eagerly inlining all 17 observer MCPtool schemas (plus any Google MCPs) into every request prefix —~+21K tokens/turn. The override re-enables lazy loading; observer'sproxy forwards tool_reference blocks byte-identically, satisfyingthe SDK's documented safety condition. Empirical: with the override +v1.7.23 defaults, the proxy is −6.9% mean cost vs no-proxy onClaude Code's reference rig (n=8 lumen refactor task, V7-22 binary).Without it, ~+9% per-turn overhead. Seesuperbased.app/docs/connect/claude-codefor the full picture.

The proxy logs every turn with the exact token counts the providerreturned, including cache-tier breakdowns (5m vs 1h ephemeral) and1h surcharges that JSONL adapters can't always disambiguate.

Verifying the install

observer doctor          # health checks: DB integrity, hook
                         # registration, MCP entries, pid bridge
observer status          # row counts + recent activity
observer tail            # live-stream captured actions

Dashboard tour

observer start opens the dashboard automatically on interactivelaunches (suppress with --no-open; default URLhttp://localhost:8081). Eighteen tabs in four nav groups (Monitor /Analyze / Optimize / Configure), each designed around one question —the tour below covers the core surfaces; Live (recent sessions with areal-time action feed), Search (full-text over captured tool outputs),and Privacy (capture map + scrub tester) are self-explanatory onceyou're in, and the Suggestions tab's advisornudges and the Patterns tab's derived habits are covered in their owndocs. Evaluating without data? Start demo mode from the emptyOverview — synthetic dataset in a temp DB, real observer.dbuntouched, one click to clear.

Overview — what's been happening?

Four headline KPI tiles (sessions, API turns, token rows, stalere-reads — each filterable by the global Window / Tool / Projectchips), cost-over-time stacked area split by billable token bucket,actions-over-time stacked by tool, top models by token volume, toptools by action count.

Sessions — what did each run actually do?

One row per session with cost, token totals (input / cache R /cache W / output), elapsed time, action count, and a model badge.Quality / Errors / Redundancy scoring columns light up onceobserver score has run. Click a row to open the per-sessionslide-over (shown below in Session detail).

Actions — the firehose, filtered

Every recorded tool call, normalized across adapters. Filter byaction type (28 categories), tool, effort, permission. Each rowexposes its target + status + raw-tool source + truncated contentpreview; click to expand to the full event with error context.

Cost — per-model breakdown with the right math

Eight KPI tiles across the billable token buckets (Net Input, CacheRead, Cache Write 5m, Cache Write 1h, Output, Reasoning, plus totalUSD and turn count). Per-model table shows the full breakdownincluding reasoning tokens (billed at output rate) and long-contextsurcharges (Sonnet 1M, gpt-5 >272K, Gemini 2.5 Pro >200K). Hoverany column header for its definition + formula.

Analysis — spending insights & efficiency signals

Twelve KPI tiles comparing this period to prior: spend Δ%, MTD vsbudget with projection bar, $/M output rate, cache savings + cacheefficacy %, high-context turn count, $/turn, burn rate ($/activehour), top model concentration %, Discovery waste $, sessionstotal. Daily-spend stacked bars with Model / Project / Tooldimension toggle, hour-of-day heatmap, period-over-period movers(top increases / decreases / new entrants), and model right-sizinghints (trivial Opus sessions that could have used Sonnet).

Tools — per-AI-client breakdown

Four KPIs (total actions, distinct tools, overall success rate,busiest tool), activity-over-time stacked area, and per-toolaction-type-mix horizontal bars (100% normalized, colored by actioncategory). Surfaces which AI client owned which kind of work.

Compression — what the proxy saved

Five KPIs: total $ saved (priced at your input rate), tokens saved,bytes trimmed, turns compressed. Savings-per-day stacked bar bymechanism (drop, trim, summary), savings-by-mechanism donut, recentevents table with original→compressed→saved + dollar impact perevent.

Cache — prompt-cache observation & forecasting

Headline cache-ratio hero (cache_read ÷ cache_write tokens) and threesibling KPIs: Cache read, Cache write, and Avoidable spend / Eventcount. Avoidable spend renders in warn tone — it's the dollar overheadof rewrites that wouldn't have happened on a perfectly cache-friendlysession. By-model and By-project tables with R%/W% mix bars + absoluteRead/Write/Events + cache Ratio + Avoidable $. Proportional Topcauses histogram (suffix_growth + hit dominate a healthy session;real invalidations render in warn tone; tools_changed on MCP togglesrenders neutral). Worst sessions table sorted by rewrite count;click-through opens the per-turn Cache panel.

How it's captured. Two paths feed the same engine, both writingto NODE-LOCAL cache_segments / cache_entries / cache_eventstables (migrations 036+037, never pushed to a Teams org server):

• Tier-1 (proxy) — point your AI client at 127.0.0.1:8820 andthe cachetrack engine reads each turn's cache_read_input_tokens +cache_creation_input_tokens envelope live. Default capture pathfor Claude Code.
• Tier-2 (transcript watcher) — feeds the same engine fromon-disk claude-code JSONL transcripts for sessions that didn'troute through the proxy. Run observer backfill --cache-rescanto retrofit history.

Enable / disable. Default-on per spec §11 (the loader merges[cachetrack].enabled = true if the section is absent). To turn off:[cachetrack].enabled = false in ~/.observer/config.toml, thenrestart observer start. Inspect engine health withobserver cache-health --json. Operator reference:superbased.app/docs/guides/cache-tracking(or docs/cache-tracking.md in the repo).

Suggestions — the advisor's quantified nudges

Default-on, fully local suggestions engine (zero LLM cost, zeronetwork): 19 detectors turn the window's captured activity intoranked, dollar- or minute-quantified recommendations — sessionballoons, idle re-cache, long-context tier crossings, trivialsessions on expensive models, cache hit-rate / cache-write waste /prefix thrash, read-heavy expensive-model sessions, effortoverprovisioning, fast-tier premium, unrecovered failures, qualityregressions, MCP schema overhead, compression off, capture withoutproxy routing, cross-session stale reads, web-search spend, spendspikes, plus a posture nudge (guard observing idle, pointing at thesurface that owns the workflow). Everycard carries its arithmetic ("show math"), a confidence score,snooze/dismiss with a 7-day cooldown, and — where a dashboardcontrol can fix the finding — a one-click action that navigates tothe right surface (writes stay behind that surface's own consentflow). CLI twin: observer advise. Config: Settings → Advisor([advisor] — evidence window, confidence/savings floors, opt-in≤400-token session-start digest).

Discovery — the waste detector

Waste $ hero (stale-read tokens × your blended input rate). FourKPIs: stale re-reads count, tokens wasted, affected files, repeatedcommands. Top files re-read table with cross-thread highlighting(when the same file was re-read from a subagent that didn't see theparent's read). Repeated-commands table with no-change-rerundetection.

Security — the guard, operable end to end

Posture tiles + a filterable verdict timeline (rule IDs resolve totheir full definitions), then the routine workflows: a consent-gatedmode control that shows the simulate evidence before you flipenforce, the enforce-readiness replay over your real history, theapprovals register (scoped, expiring — live immediately), alint-gated user-policy editor with .bak undo, budget guardrailssuggested from your own observed spend with a daily burn-down meter,MCP pin approvals, and one-click compliance evidence downloads.

Settings — every config knob, editable

Schema-driven forms for every config section — Watcher, Freshness,Retention, Hooks, Proxy, Compression, Intelligence, Advisor, Cachetracking, Secrets scrubbing, MCP, Profiles, Org share, OTel — withhonest reload semantics per section: pricing and profile changesapply hot, MCP applies to the next AI session, restart-gatedsections raise a persistent restart-pending banner that names theexact command and clears only when the daemon actually restarts.Alongside the forms: a Connected tools panel (per-tool statusmatrix, consent-gated setup wizard, Launch button), a Healthpanel (the observer doctor checks + recent failures), theBackfill panel (every mode click-to-run with streamed output +full rescan), a Storage panel (per-table DB size breakdown withindex/FTS bytes folded in, vacuum + online backup as click-to-runjobs, documented manual restore — CLI twin observer db stats|vacuum|backup), and a config-file card with one-click .bakrestore.182 baked-in default models; pricing "Override" prompts auto-fillfrom the default.

Live — what's running right now

Every session with activity in the last 15 minutes, refreshing on a5-second tick: lifetime cost, tokens, turns, and a streaming actionfeed per session. The fastest way to confirm the proxy is capturingwhile you work.

Trajectories & Evals — observe an LLM app you host

General observability — Plane A. Distinctfrom the coding-agent usage above: this observes an LLM applicationyou run at the admin level, whose end-users route through Observer.

Point any OpenTelemetry-instrumented app or agent at the local OTLPendpoint (or route it through the proxy) and its trace/span graph iscaptured into node-local obs_* tables — enriched with the proxy'sexact cost and cache numbers wherever a span matches a proxied turn(the join no external observability tool can make). You viewtrajectories and eval-run health on the admin dashboard (web2, theTrajectories nav group) — the node/developer dashboard stays focused onyour own coding-agent activity. Capture is local and node-opt-in, andthe same plane adds per-end-user budgets + an input-admission guardrailjudged by a local, custom-remote, or gateway LLM (seesuperbased.app/docs/getting-started/two-planesfor the Plane A guardrail model).

Session detail — drill into one session

Click any session row → slide-over with an action-type breakdowndonut, a token-bucket bar (net input / cache R / cache W / output),and the models used. It also carries the next-message costpredictor (a low / typical / high band over the session's likelyturn fan-out) and, for proxied subscription sessions, the 5-hour /weekly limit gauge read from the provider's own rate-limit headers.

Scroll down for the full per-message timeline — every upstreamAPI turn with its model, token buckets, per-turn cost, and the toolcalls nested inside it, each row expandable. The same panel surfacesfrom Actions when you click a session pill.

MCP server — 25 cross-tool intelligence calls

Opt-in. The MCP server is not active until you runobserver init (or observer init --claude-code / --cursor /--codex). That command writes entries pointing at the observerbinary into each AI tool's own MCP config file(~/.claude.json, ~/.cursor/mcp.json, ~/.codex/config.toml).The MCP server then runs as a stdio subprocess spawned by your AItool — its lifecycle matches the AI tool's, and it never opens anetwork port. observer start alone does NOT register or launchthe MCP server; it can be skipped entirely viaobserver init --skip-mcp if you want hooks-only capture.

Once registered, every connected AI tool can query the observerover MCP/stdio: 21 tools are always registered, and 4 moreregister conditionally (only when the capability they depend on —the proxy stash, or the codeintel index — is actually configured):

Operator note for Claude Code via observer's proxy. When you setANTHROPIC_BASE_URL=http://localhost:8820, Claude Code's SDKdisables ToolSearch:optimistic deferred MCP loading — observer'stool schemas (plus any Google MCPs you've registered) end upeagerly inlined into every request prefix, ~+21K tokens/turn. SetENABLE_TOOL_SEARCH=true in the same shell to recover lazyloading; observer's proxy forwards tool_reference blocksbyte-identically, satisfying the SDK's documented safety conditionfor the override. Seesuperbased.app/docs/connect/claude-codefor the full picture.

Knowledge captured from one tool benefits all the others working onthe same project — data is organized by git root, not by tool. Aread by Claude Code becomes a freshness signal for Codex; a Cursorcompaction is visible from Cline.

API proxy — accurate tokens, compression, stash

The proxy is the home of three features that only exist when your AIclient routes through it. None of them run on the watcher / observer start ingestion path — compression and stash live in the requestpath because that's the only place where bytes can be rewrittenbefore they reach the upstream provider.

When you point your AI tool at http://localhost:8820, the proxy:

1. Forwards your request to your chosen upstream (Anthropic orOpenAI). The destination is the same provider URL your AI clientwould have called directly; no data leaves your machine thatwasn't already going to that provider. Your API key is yours —the proxy reads it from the inflight headers, never stores it.
2. Records the exact token counts the provider returned (cache5m vs 1h split, long-context tier triggers, reasoning tokens)into the api_turns table — more accurate than parsing theJSONL the AI tool wrote.
3. Compresses the conversation before forwarding(importance-scored, prefix-stable for cache alignment) — thebiggest lever for keeping long sessions inside rate-limitwindows. Opt-in: flip [compression.conversation].enabled = true (or the Settings → Compression toggle); once enabled, tunedper-tool profiles apply automatically (see below), with the safeper-type set (compress_types = ["json","logs","code"]) onAnthropic traffic. Compressed events land in thecompression_events table and surface on the Compressiondashboard tab. Empirical on Claude Code lumen rig (n=8, V7-22):−6.9% mean cost vs no-proxy, CV 7.6%, zero tail outliers.
4. Stashes large tool outputs the compressor hides, so theoriginals stay retrievable via the retrieve_stashed MCP tool(only registered when stash is configured). Off by default —stash markers break Anthropic's prefix cache (V7-25 n=1measurement: +25% cost, cache_creation_input_tokens doubled).Operators who want stash on a workload should A/B beforecommitting.

Three compression layers, each independently toggleable:

• Shell output filters — RTK-style truncation of large bash /git / go test / docker / kubectl / cargo / pytestoutputs inline before they hit the LLM context. Runs on hook /observer run paths; does not require the proxy.
• Tool output indexing — every tool call's output indexed intoFTS5; large outputs trimmed to a 2KB excerpt cap so the indexstays compact and search_past_outputs stays fast. Runs on thewatcher path; does not require the proxy.
• Conversation compression — proxy rewrites large tool_resultblocks before forwarding upstream. Proxy-only — there is nonon-proxy path for this layer, and the stash that backsretrieve_stashed is wired here too.

Trade-off if you skip the proxy: you still get full hook + JSONLingestion, the dashboard, MCP (if registered), and shell+indexingcompression. You lose proxy-grade token accuracy, conversationcompression, and the stash. For rate-limited plans (Claude Teams 5h/7dwindows), conversation compression is usually the difference between"finishes the task" and "hits the limit."

Choosing a compression mode (Anthropic vs Codex)

mode is a profile parameter — each shipped profile (next section)already pins the right one, so most users never set it by hand; thematrix below is the why. It behaves differently per provider. Per-typetool_result compression runs in every mode; mode only changes how messagesare dropped and whether an Anthropic cache_control marker is injected.

The shipped default is cache_aware (token is just the internal fallback whenmode is empty). The cache/cache_aware strategies exist for Anthropic'scontent-hash prefix cache (cache_control is an Anthropic Messages APIconcept). OpenAI/Codex prompt caching is automatic and server-side — thereis nothing to mark or tune, so the proxy's OpenAI path is mode-agnostic (thedefault cache_aware simply behaves like token there).

Profiles — the right parameters per tool, automatically

Compression parameters ship as profiles: named parameter setsresolved per traffic class at the proxy boundary (formerly "recipes",which had to be hand-picked per daemon — and applied to all traffic,mis-tuning whichever tool the recipe wasn't written for). Enablecompression once and the tuned parameters apply per toolsimultaneously:

"variant" in codex-variant means the -codex model variant(gpt-5.3-codex, anything *-codex*), not a variant of the codexCLI — both codex profiles are for the codex CLI.

Working with profiles:

• Inspect: observer profile list (assignments + sources),observer profile show codex-safe (the resolved TOML).
• Reassign per traffic class or per tool: observer profile assign openai codex-variant, observer profile assign tool:cline codex-safe — or Settings → Profiles in the dashboard.
• Custom profiles: observer profile create mine --from claude-code, then observer profile set mine compression.conversation.target_ratio 0.9 — or the Settings →Profiles editor.
• Per-project overrides: a repo-level .observer/config.toml([profiles] + [compression] keys only) picks different profilesor turns compression OFF for that repo's traffic — never on(untrusted-repo guard).
• Hot for new sessions: profile edits and assignment changesapply without a daemon restart. The master enabled switch itselfis restart-gated — the dashboard banner says so honestly.
• observer start --recipe <name> survives as a deprecated aliasthat pins one profile for all traffic.

Measured effects vary by traffic shape and workload — from thedouble-digit savings above to a since-fixed configuration that onceregressed cost on a different binary. Historic marketing claims of aflat savings percentage are retracted; compression ships opt-in anddefault-off precisely because the right call depends on your owntraffic — A/B it before relying on any number here. Full knobreference (stash, rolling-summary's per-provider modelsplit, compaction):docs/compression-modes.md. Then routeyour client through the proxy — the one-click button path in theFirst-run walkthrough — and restart it.

Architecture

┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│ Claude Code  │     │    Cursor    │     │    Codex     │    ... 26 adapters total
└──────┬───────┘     └──────┬───────┘     └──────┬───────┘
       │ JSONL              │ hook events        │ rollout files
       ▼                    ▼                    ▼
┌────────────────────────────────────────────────────────┐
│              fsnotify Watcher + Adapters               │
│      (one parser per platform, normalized output)      │
└───────────────────────────┬────────────────────────────┘
                            │ ToolEvent / Action / Session
                            ▼
┌────────────────────────────────────────────────────────┐
│   SQLite (WAL, pure-Go via modernc.org/sqlite)         │
│   actions · sessions · projects · file_state ·         │
│   api_turns · token_usage · failure_context ·          │
│   compaction_events · action_excerpts (FTS5)           │
└───────────────────────────┬────────────────────────────┘
                            │
       ┌────────────────────┼────────────────────┐
       ▼                    ▼                    ▼
┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│  Dashboard   │     │  MCP Server  │     │  API Proxy   │
│ HTTP+/api/*  │     │  stdio · 25  │     │ Anthropic +  │
│   18 tabs    │     │    tools     │     │    OpenAI    │
└──────────────┘     └──────────────┘     └──────┬───────┘
                                                 │
                                                 ▼
                                           upstream API
                                           (traffic passes
                                           through verbatim,
                                           with token tracking
                                           + optional compression)

Adapters parse each platform's session format into a unifiedAction row. The cost engine reads api_turns (proxy, accurate)and token_usage (JSONL adapters, approximate) and deduplicatesper-turn via the upstream request_id ↔ source_event_id match.The MCP server exposes the same database. The dashboard pulls JSONfrom the same /api/* endpoints the MCP tools use.

Teams & Org Visibility

For teams, an optional observer-org server gives admins org-widevisibility — per-developer and per-team spend, project rollups, and budgets —without changing the solo-local experience at all. It is purely additive: adeveloper who never enrols sees a byte-identical local tool. Developers opt inwith observer enroll --link <magic-link>, after which the agent pusheshash/metadata-only rollup rows (signed, gzipped, deduplicated) to theserver. The default v1.8.0+ posture ships sha256 hashes of command targets,filesystem paths, and git remotes — never the raw values; a per-node opt-in([org_client.share].full_content = true) lets a developer choose to sharefull content, but the org admin cannot flip this remotely.

Authentication is SAML SSO for humans, SCIM 2.0 for provisioning, and Ed25519bearers for agents; no prompts, command bodies, full assistant responses, orfile contents ever leave the machine — even in the opt-in full-content mode,only the small set of fields the agent already stores in metadata-only modegets expanded. The server ships as a signed Docker image(ghcr.io/marmutapp/observer-org) and a Helm chart (charts/observer-org/).

Local-in-5-minutes: observer-org quickstart brings up a dev compose stack,provisions an admin, mints an enrolment token, and prints a ready-to-sharemagic link. observer org status / preview / backfill give the developerfull visibility into what their agent is shipping.

Independently, each agent can export per-turn LLM spans to your ownOpenTelemetry collector (gen_ai.* + sbo.* attributes, off by default).

• Getting started — observer-org quickstart for local-in-5-minutes; production SAML/SCIM bring-up.
• Architecture — components, runtime dataflow, the v1.8.0 privacy posture, source map.
• Operations — backup/restore, key rotation,upgrades, troubleshooting, scrub-content for legacy DBs.
• Local test plan — an internal 7-phase evaluator plan covering localbring-up end to end in ~15 minutes (source-repo reference, not shipped inthis distribution).

Security & control layer (guard)

The guard evaluates every captured agent action against a table-driven policy— built-in rules (destructive commands, project boundaries, secrets egress,MCP pinning, taint/dataflow, budgets) plus your own TOML rules — records eachverdict in a hash-chained tamper-evident audit table, and blockspre-execution on the channels that support it (Claude Code and Cursor hooks,the proxy egress path, plus native permission rules it compiles into eachclient's own config). It ships observe-only by default: nothing blocksuntil you flip enforce, and observer guard simulate --since 168h replaysyour real history against current policy first so you know exactly whatenforce would have done.

Teams installs can distribute an Ed25519-signed org policy bundle that mergesas a strictness floor (escalate-only — the server can never weaken nodepolicy), with fleet rollups and RBAC on the org dashboard. Audit rows exportas JSONL/CEF for SIEM ingestion, and observer guard report renders acompliance evidence pack mapped to SOC 2 / NIST 800-53. The no-networkinvariant holds: nothing leaves the machine unless you opt into Teams push,the OTel feed, or the cloud alerting tier individually.

Optional process observability ([observer.process], opt-in, off bydefault) attaches the OS-level process tree — Linux eBPF or Windows ETW —beneath each captured session, for the runtime side effects hooks alonecan't see (a spawned binary, an unexpected child process).

Alerting. Guard verdicts and budget/obs-alert crossings can push outthrough desktop toast notifications ([guard.alerts] desktop = true) andoutbound webhooks — generic, Slack, Discord, or PagerDuty([[guard.cloud.webhooks]] for guard events; the org server's per-budgetand per-obs-alert-rule webhook columns for spend/eval alerts) — each behindits own opt-in, routed through one egress worker with an endpoint allowlistand a payload cap. Nothing fires until you configure it.

Every routine guard workflow also runs from the dashboard's Security page(mode control with simulate evidence, enforce-readiness replay, approvals,lint-gated policy editor, budget guardrails, evidence downloads) — see theDashboard tour above.

• Operator guide —concepts, modes, the observe→enforce path, Teams policy merge, and thehonest "what guard does NOT do" list.
• Rule catalog, policy authoring, compliance mapping — every built-inrule ID, the TOML matcher vocabulary with a worked 7-recipe cookbook, andthe SOC 2 CC-series + NIST 800-53 AU-2/AU-3/AU-9/AC-6 evidence-packmapping ship as docs/guard-rules.md, docs/guard-policy-authoring.md,and docs/guard-compliance.md for anyone building from source.

CLI reference

observer <command> --help for full flag listings. Bare observerprints a status welcome (daemon up? dashboard URL? next step) insteadof the help wall; observer --help keeps the complete list.

Configuration

Default location: ~/.observer/config.toml. Override with--config. A minimal config:

[observer]
db_path = "~/.observer/observer.db"
log_level = "info"

[proxy]
enabled = true
port = 8820
anthropic_upstream = "https://api.anthropic.com"
openai_upstream = "https://api.openai.com"

[intelligence]
monthly_budget_usd = 100   # surfaces on Analysis tab; 0 hides

[compression.shell]
enabled = true
exclude_commands = ["curl", "playwright"]

[compression.indexing]
enabled = true
max_excerpt_bytes = 2048

[compression.conversation]
enabled = false            # opt-in; modifies request bodies in flight
mode = "cache_aware"       # "token" | "cache" | "cache_aware" — see "Choosing a compression mode" below
target_ratio = 0.85
preserve_last_n = 5
compress_types = ["json", "logs", "code"]

# Per-model pricing overrides. Only specify what differs from baked-in.
[intelligence.pricing.models."claude-sonnet-4-6"]
input = 3.0
output = 15.0
cache_read = 0.30
cache_creation = 3.75
cache_creation_1h = 6.00

# Long-context tier example (Anthropic Sonnet 1M, gpt-5 >272K,
# Gemini 2.5 Pro >200K). When prompt exceeds the threshold, every
# rate is replaced with its long_context_* counterpart for that turn.
[intelligence.pricing.models."claude-sonnet-4-5"]
input = 3.0
output = 15.0
cache_read = 0.30
cache_creation = 3.75
cache_creation_1h = 6.00
long_context_threshold = 200000
long_context_input = 6.0
long_context_output = 22.50
long_context_cache_read = 0.60
long_context_cache_creation = 7.50
long_context_cache_creation_1h = 12.00

Every key has a TOML environment-variable override:OBSERVER_<SECTION>_<KEY> (uppercased, underscores). Nestedsections join with extra underscores:OBSERVER_COMPRESSION_CONVERSATION_ENABLED=true.

The Settings tab in the dashboard provides a fully-editable visualeditor for everything in config.toml — pricing and profile changesapply hot, MCP settings apply to the next AI session, andrestart-gated sections raise a persistent banner that names theexact restart command and clears itself once the daemon actuallyrestarts (the daemon never restarts itself).

Post-upgrade hygiene + recovery

After upgrading the binary, or whenever the Actions tab looksgappy (watcher fell behind, daemon restart with stale state,fsnotify event drops):

observer backfill --all

Re-walks every JSONL the adapters know about from offset 0, thenruns the surgical column-update backfills (cache-tier, message-id,etc.) on top. The (source_file, source_event_id) UNIQUE indexkeeps the pass idempotent.

The dashboard surfaces a top-of-page banner whenever the watcher'sparse cursor for any session file is more than 10 KB behind theon-disk size, so silent data loss has a visible signal.observer backfill --help lists every supported flag.

Build from source

git clone https://github.com/marmutapp/superbased-observer
cd superbased-observer
make build        # builds bin/observer + bin/antigravity-bridge.exe
make test         # full test suite (race detector enabled)
make all          # fmt + vet + lint + test + build

Requirements: Go 1.22+. No CGO. SQLite via modernc.org/sqlite(pure Go). golangci-lint optional for make lint. Dashboardsource under web/ (React + TypeScript + Vite + Tailwind); thecompiled bundle is committed tointernal/intelligence/dashboard/webapp/dist/ so a contributorwho only touches Go can make build without Node.

If you edit web/src/:

# Hot-reload dev loop (Vite serves :5173, proxies /api/* to observer)
./bin/observer dashboard --addr 127.0.0.1:8081 &
cd web && npm install && npm run dev

# When done, rebuild the embedded bundle and commit
make web-build
git add internal/intelligence/dashboard/webapp/dist web/dist

make web-build regenerates both web/dist and the embedded copy.Requires Node 22 LTS.

CI gates

Every PR + push to main runs .github/workflows/ci.yml:

• frontend — npm ci → typecheck → build → dist-consistencycheck (asserts the committed embedded bundle matches a fresh build).
• go — go vet → go test -race → cross-compile.

Tag pushes (v*) trigger .github/workflows/npm-release.yml:builds the frontend once, cross-compiles five platform binaries,ships them to npm, and creates a GitHub Release.

Contributing

PRs welcome. Table-driven tests in every package; >80% coverage oncore packages (cost, freshness, adapters, store). Run make testbefore submitting; make all locally to catch fmt/vet/lint issues.Conventional commits (feat: / fix: / chore: / docs: /test: / refactor: / perf:).

For larger changes, open an issue first to align on scope. Adaptercontributions (a new AI coding tool to support) are particularlywelcome — see existing adapters in internal/adapter/ for thepattern.

License

Apache 2.0 — see LICENSE.