vinvai

What you get, in one loop

Run · Test · Find — then Prove. Point Vinv at a Python repo; it does the rest — no code changes, no API keys.

• 🏃 Run — brings every service in your repo up under tracing with zero edits, capturing timings, arguments, return values and call trees from the real run.
• 🧪 Test — drives real requests through every endpoint (valid, boundary, negative, authenticated) and banks each response as a regression case.
• 🔎 Find — surfaces what actually broke or slowed down — server errors, crashes, latency hotspots and dead code — each tied to the exact source line.
• ✅ Prove — hands that evidence to the agent you already use (Claude Code, Cursor, Copilot…), then verifies its fix against acceptance tests written before the fix that it never sees. A "faster" change that alters any output is auto-reverted.

_{Your agent is the only LLM — no new bill, no model picker, no provider keys. Everything runs on your machine.}

The problem

84% of developers now use or plan to use AI coding tools. More of them actively distrust the output (46%) than trust it (33%) — and distrust nearly doubled in a year (Stack Overflow 2025, 49k developers). You know why: the agent edits the wrong handler, invents return shapes, then grades its own homework while the server won't even start.

Or it gets stuck — test fails, agent edits the same function, test fails the same way, agent edits it again, burning your context window on "let me verify." Anthropic's own research documents agents "stuck in loops, repeating the same failed approach" when they lack codebase context.

Both failures have one root cause: the agent has never watched your code run. It argues from static text.

The industry automated writing and left proving entirely manual. Vinv automates the proving — and only then the finding and the fixing.

Context beats model size

Receipts first — then how the loop produces them.

Vinv found four bugs and one performance problem in fastapi/full-stack-fastapi-template (~44k★). Same five issues, same prompts, Vinv grading every run:

One trial per condition — a demonstration, not a benchmark. Blind, the commodity model scored zero. Hand it the failing frame, the caller chain, and the real argument values, and it beats a stronger model guessing from static code. The evidence is what moved, not the weights.

On that same pristine template the optimization loop later detected — from live traces alone — that the app's default database pool makes requests queue for connection checkouts under concurrent load, dispatched the pool-sizing fix, and proved it: sustained-load median 75.6ms → 41.2ms, 45.4% faster (95% CI [36.3%, 45.8%]), responses byte-identical. Two earlier attempts whose measurement windows couldn't certify the win were auto-reverted — the accept landed only when the evidence did.

Same discipline, upstream on Hugging Face

Pointed at huggingface/smolagents (~28.5k★) — a public Apache-2.0 agent framework, no affiliation — the allocation loop found and proved a fast-path in sanitize_for_rich. Benchmarked with tracemalloc on a realistic 4 KB log line: transient per-call allocation 36.27 KB → 0.00 KB (~37,137× less); end-to-end on log_task, ~615.7 KB → ~125 B across 3 calls; output byte-identical across 2,015 inputs. Upstream as PR #2572.

That is how come after it: oracles find the waste, your agent proposes the edit, paired-bootstrap + byte-identical replay decide accept or revert, and only then does anything go upstream. The rest of this README is the machinery behind those receipts.

Install

Use it from your IDE, your CLI, or any MCP-compatible agent. Vinv isn't another coding agent — it's the runtime-evidence layer underneath the one you already use, exposed as MCP servers (vinv-index, vinv-runtime, vinv-exercise) that Claude Code, Cursor, Codex, Copilot and Windsurf pick up automatically.

One click from the marketplace — Open VSX — or straight from your editor's CLI:

_{First run builds the engines — about 4 minutes: it compiles the Rust index and fetches a one-time ~500 MB local embedding model (uv and Rust required). First trace lands about a minute after that; everything after is seconds.}

git clone https://github.com/VinvAI/VinvAI ~/.vinv/engines && cd ~/.vinv/engines && ./install.sh

_{Windows (PowerShell):}

git clone https://github.com/VinvAI/VinvAI $HOME\.vinv\engines; cd $HOME\.vinv\engines; .\install.ps1

Under the hood: the oracle roster

The Test stage above isn't one tester — it's a set of oracles, each hunting a different class of defect, all writing into the same findings and the same fix-dispatch path. You never need to think about them to use Vinv; open this if you want the full list and how the budget is spent.

The dispatcher is real, and it is a bandit. exerciser campaign allocates one budget across every armed oracle by Thompson sampling over (target × technique × oracle) — rather than driving each one exhaustively. Cost is measured (wall-clock normalized to probe-equivalents plus subprocesses spawned), so an oracle that takes forty seconds to find what another finds in one loses. Credit is paid once per defect signature, within a run and across runs, so a deterministic oracle can't re-earn credit for the same bug forever. Posteriors persist in campaign.json — which technique pays on your repo is learned.

The campaign dispatches six oracles (default_runners): crash/function, differential, fault, concurrency, HTTP, and environment — each armed only when it applies to your repo (no --base-url and the HTTP oracle stays dark; no boundaries and the fault oracle does). The dead-code and runtime-analysis families (leaks, hotspots, cache candidates) and the golden-I/O baselines in the table below run from the editor and the regression path, not this budget loop.

Unverified code runs behind a containment ladder: a kernel-enforced OS sandbox (sandbox-exec / bwrap / unshare) where the host offers one, otherwise a process shim — always with a disposable repo copy, redirected HOME/TMPDIR, blocked network and subprocess spawning. Tier is decided by a probe that verifies a write outside the root really failed, never by a binary being on PATH. Postgres, Redis and S3 are substituted inside the jail so code that needs them runs instead of failing to connect.

Dead code, from what actually ran

Static tools (Vulture, deadcode, Knip, ts-prune) can only prove "nothing statically references this." They can't see dynamic dispatch, feature flags, registries or environment drift — so they emit candidates a human has to adjudicate, which is why the cleanup never happens.

Vinv can say something they can't:

"No capture ever executed this. Here is what still references it, here is the traced neighbourhood it would wire back into, and here is what your agent thinks it is."

The unit is a section — a connected island of untraced symbols — not a lint row, because dead code is almost never one function. Each section gets:

• Reachability evidence — REACHED FROM LIVE CODE (executing code references it; the path was never taken — usually a guard or an unshipped feature) versus NO REFERENCES at all. Opposite verdicts, and a filtered canvas can't tell you which.
• The live neighbourhood — Personalized PageRank seeded at the section's symbols over the code graph, keeping the highest-mass traced symbols. HippoRAG's retrieval idea, run over a graph Vinv already has. That's the place an integration would wire into.
• Your agent's verdict — integrate · reimagine · delete · keep · unclear, each with what it does, why nothing reaches it, and what breaks if it's removed. Sections travel five to a prompt so the agent can say "this is the older copy of the section below" — a judgment a per-section run structurally cannot reach.

Two things it refuses to do: call anything dead with no trace on disk (with zero captures every symbol is untraced, so the list would be your codebase), and drop anything silently — both caps are recorded as lineage, so "12 sections" is distinguishable from "12 is the cap and 300 were dropped".

And the second-order reason to care: dead code makes your coding agent worse. Every unused module competes for the context window, burns tokens, and offers wrong patterns to copy.

The four acts

01 FIND — what actually ran

Static scanners guess. Vinv records a real run and ranks what failed, what never executed, and what was slow. No "possible issue" — every finding names a symbol, a line, and the trace behind it.

# with a service up — Auto-Pilot's path: plan every endpoint, then drive them
exerciser plan <repo> --base-url http://127.0.0.1:PORT && exerciser run <repo> --base-url http://127.0.0.1:PORT
# allocate budget across armed oracles (HTTP + the rest that apply to that run)
exerciser campaign <repo> --budget 20

02 FIX — through the agent you already pay for

Findings become evidence packs and go to your own agent — Claude Code, Codex, Cursor, Gemini CLI, Copilot Chat, Windsurf Cascade. Your agent is Vinv's only LLM. No new bill, no model picker, no provider keys.

The pack is composed from a context graph: your code, your traces, and the metrics derived from them, joined on the exact function that handled each request. The artefacts are commodities; the join is not.

03 PROVE — or revert it

Replayed start. Live port. Acceptance tests authored before the fix, stored outside the workspace under an opaque token, and required to fail deterministically twice on the pre-fix code — the SWE-bench fail-to-pass discipline. A test that passes on broken code is discarded.

Around that: a static pre-gate (every changed .py must ast.parse before a test budget is spent), a deterministic anti-cheat diff audit over the snapshot ref plus untracked files (test edits, except swallows, interpreter shadow modules, .vinv tampering — hard flags block eligibility outright), a bounded LLM judge that can push toward scrutiny but can never rescue a failed gate, and an advisory mutation smoke whose survivors are never revealed to the fixing agent — the Goodhart guard.

One click reverts everything an episode touched, untracked files included.

04 LEARN — from what survived

Two ledgers, both local, neither uploaded.

Context-pack composition is a factored 2² arm grid (graph-slice depth × runtime-evidence inclusion). Selection is Thompson sampling with an ε-floor mixture, and the exact mixture propensity is logged with each decision — the requirement for unbiased IPS/SNIPS/DR later. The ε-floor decays but never reaches zero, so importance weights stay bounded. Posteriors count only objective episodes — a user abort or an "approve as done" click is not evidence about arm quality. Attribution is a COMA-style counterfactual per feature over the grid of posterior means: "did runtime evidence help" is a computed number, not a claim.

Retrieval serving is a separate ε-greedy bandit (over the top-k action set) with its own ledger. A candidate config is promotable only when all hold: ESS ≥ 25, n ≥ 40 joined samples, ≥ 8 logged pulls per compared action, BCa-bootstrap 95% lower bound of the doubly-robust delta ≥ 0, zero clipped weights, and no epoch contradicting the pooled delta (the Simpson-artifact guard). A promoted policy then serves at 5% canary only, and three consecutive negative canary rewards roll it back automatically.

Measured on this repo's own ledger (800 logged decisions, 770 joined, 12 index epochs — docs/learning.md):

The gate admitted exactly the measured winner and blocked both the uncertain and the unsupported candidate. That is what "closed-loop" is being asked to mean here — and the parts that are not online learning (the referee's thresholds, the oracle catalogue, the fault shapes) are fixed policy, deliberately.

If any of these is your open tab

What Vinv does

Give your coding agent runtime context — one loop, these capabilities:

• Semantic code search — ask by meaning, get ranked symbols with def bodies and line numbers, embedded by a local model (no cloud keys).
• Code Graph — a persistent map of every symbol and call edge, updated incrementally on save, with a live runtime overlay.
• Runtime tracing — zero-edit runtime tracing for AI coding agents: timing, memory, args, returns, errors — per call, joined to source.
• Rank suspects — on any failure, symbols ranked by fault-localization score over real pass/fail requests, error messages attached.
• Verified fixes — verify AI-generated code actually works: replayed start, live port, acceptance tests the agent never sees. One click reverts everything an episode touched.
• Dead code sections — "View Dead Code" explains every untraced island. Sections split into no references and reached from live code but never taken, each with the callers that still point at it and a keep-or-cut verdict with its reasoning.
• Recoverable time — latency hotspots ranked by the milliseconds you would actually get back, each dispatched as a predicted-then-proven optimization instead of a guess about what is slow.
• Ask Vinv — ask anything about your running system in plain English; every answer cites the exact trace spans and source lines it came from, and a deterministic critic blocks any claim the evidence can't back — grounded Q&A, not confident guessing.
• Behavior exerciser — Vinv doesn't wait for traffic: it drives every discovered service endpoint itself, picks strategies with a Thompson-sampling bandit rewarded by oracle violations first and new coverage only as a bonus, and turns every response into a permanent regression case.
• Journey — one walkthrough of everything verified: every service, then every endpoint's call tree, latency flamegraph, and the exact inputs → outputs exercised — with a form to add your own test inputs that the engine replays forever after.
• Auto-Pilot & the red ring — one click drives discover → set up → trace → exercise → fix → verify until green or budget; when new trace errors land, the fix episode is already dispatched by the time you see the red ring in the graph. The budget is yours: set attempts per service in Configure, and when a run exhausts them Vinv asks whether to grant more instead of quietly giving up.
• Agent babysitting — a doom-loop guard (token-set self-similarity) catches a repeating agent, an adaptive silence watchdog catches a hung one, and "Dispute a Verified Fix" keeps even the verifier accountable.
• Findings — what Vinv found and what it fixed, with the statistical evidence: issue clusters, optimization episodes with paired-bootstrap confidence intervals, regression diff kinds, and a machine-readable findings.json your agent can consume directly.

Honest scope: Python first — services and APIs. Auto-Pilot discovers runnable services, brings them up under tracelens, and exercises their HTTP (and related) entrypoints. Other languages get the index, graph and grounded QnA, but no runtime evidence yet. TypeScript and Go next.

Why agents don't reward-hack under Vinv

Vinv ties every runtime trace to the exact code segment that produced it and hands your agent a context graph built from that join — so the agent argues from evidence, not vibes. And when the agent claims victory, Vinv doesn't take its word:

• Acceptance tests are authored before the fix, stored outside the workspace under an opaque token, and must fail deterministically twice on the broken code — it can't train to the test, and a test that passes pre-fix is thrown away.
• A "faster" fix that changes any observable output is auto-reverted — the behavior suite must replay byte-identical, and the speedup's paired-bootstrap 95% CI must exclude zero. Faster-but-wrong never lands.
• Deliberate 4xx rejections aren't "errors" to fix — the defect classifier knows the difference between a service saying no correctly and a service breaking, so the agent is never handed a fake goal it can only game. Same rule in the newer oracles: a sandbox that refuses nonlocal is enforcing a documented limit, not failing.
• Silent-wrong-value findings dispatch with value-shaped criteria — for differential-mismatch, fault-divergence, concurrency-divergence, invariant-violation and baseline-degraded, "these calls no longer raise" would be vacuous, because the target never raised. The criterion is the value.
• When two attempts stop making progress, a Nash-bargaining stall judge decides — continue only if both an explorer stance and an auditor stance strictly prefer it to asking you. Otherwise you get a judgment panel, not a token bonfire.

Exerciser on the FastAPI template

Traffic only shows you the code paths users happen to hit. The behavior exerciser drives the rest — same fastapi/full-stack-fastapi-template, one run:

The authenticated sweep (every endpoint replayed under credentials the login scenario captured, with freshly created resource IDs fed to the by-id endpoints) surfaced four real bugs that anonymous traffic can never reach:

1. GET /api/v1/users/ → HTTP 500 — an invalid email stored by an unvalidated private endpoint poisons response serialization
2. POST /api/v1/private/users/ → IntegrityError escapes as a 500 — email: str instead of EmailStr, no duplicate guard
3. POST /api/v1/utils/test-email/ → HTTP 500 — assert settings.emails_enabled crashes instead of degrading
4. POST /api/v1/password-recovery-html-content/{email} → connection killed — unsanitized header rendering

The harness then fixed all four, and the regression suite now distinguishes your code regressed from the test engine's own leftover data changed the world (the state ledger) — so a re-run doesn't cry wolf. Phantom perf regressions are gone too: a latency diff must survive a median of 5 replays before it's reported.

Works with your agent

Vinv is an MCP server for Claude Code and Cursor — and every other MCP client you already use. One command (Register Vinv MCP in Agent Tools) writes the servers into every agent it detects:

Registration is idempotent and never commits secrets. The servers (vinv-index, vinv-runtime, vinv-exercise) launch over stdio via the editor's own runtime.

• Claude Code — ~/.claude.json, project-local scope (no trust prompt). Verify: claude mcp list shows the Vinv servers.
• Cursor — <repo>/.cursor/mcp.json. Verify: Settings → MCP shows them green.
• Codex CLI — ~/.codex/config.toml under [mcp_servers.vinv-index] / [mcp_servers.vinv-runtime].
• Copilot Chat — native VS Code MCP provider (auto), .vscode/mcp.json on older builds.
• Windsurf Cascade — ~/.codeium/windsurf/mcp_config.json.
• Gemini CLI — dispatch works out of the box; for MCP tools, add the same stdio servers to ~/.gemini/settings.json.

Your agent is also Vinv's only LLM — every analysis step routes through the coding-agent CLI you already pay for. No provider keys, no model picker.

Agent without Vinv vs with Vinv

Proven on itself

Vinv's release gate is Vinv — these numbers come from running the loop on this repository:

How it works

flowchart LR
  T[Trace] --> I[Index] --> S[Serve MCP] --> V[Verify] --> L[Learn] --> T

1. Trace — run your Python service under the bundled tracer: no SDK, no code changes. (No service? The function, differential, fault, concurrency and environment oracles need none.)
2. Index — every function embedded locally into a semantic index + call graph.
3. Serve — MCP servers hand the evidence to your agent.
4. Verify — replayed start, live port, acceptance tests generated before the fix.
5. Learn — propensity-logged decisions; retrieval and pack composition update only on off-policy-evaluation wins, behind a 5% canary with automatic rollback.

No black boxes — every decision Vinv makes has a published method behind it, and each one exists to keep the loop honest, not clever:

The full learning walk — reward, propensity, gating math, with file:line for every claim — is docs/learning.md. The test ontology is docs/testing-ontology.md.

Vinv indexes the code and generates — from your own run — the traces and the metrics derived from them, then ties all three to the exact function that handled each request. The artefacts are commodities; the join is not. Auto-Pilot drives the whole loop unaided: discover services → set up via your agent → start under tracing → exercise → fix → re-verify, until green or budget. Layout: extension/ (editor UI + MCP servers), index/ (Rust semantic index), embedder/ (local CodeRankEmbed sidecar), tracelens/ (zero-edit tracer), exerciser/ (the oracle swarm + campaign bandit), identification/ (trace↔source join), handbook/ · bringup/ · goal/ (discovery & episodes), tests/e2e/ (planted-bug golden test). Python engines are one uv workspace.

After install: the things to try

1. Hunt without a server — exerciser campaign <repo> --budget 20. No service, no --base-url: the function, differential, fault, concurrency and environment oracles do the work, and the bandit reports which technique paid.
2. Exercise your API — exerciser plan <repo> && exerciser run <repo> --base-url http://127.0.0.1:PORT (or let Auto-Pilot's exercise phase do it). An environment canary first dry-runs your login chains and tells you loudly if the database was reset or credentials unseeded — no more silently-401 runs.
3. Find your dead code — Command Palette → "Vinv: View Dead Code": every untraced section, split into never-referenced vs reachable-but-never-taken, with the live callers that still point at it and a keep-or-cut verdict.
4. Walk everything — "Vinv: Open Journey". Overview first (services, coverage, open issues), then Next/→ through every endpoint: call tree with live runtime, flamegraph, and the exact inputs → outputs driven. Hover anything cryptic — every marker explains itself.
5. Add your own test input — on any Journey endpoint step, fill body/params/expected status and hit Add input. It lands in the same plan layer the AI-authored scenarios use, runs with the endpoint's auth setup on the next exercise, and becomes a permanent regression case.
6. See what got fixed — "Vinv: Open Findings": issue clusters, optimization episodes with their confidence intervals, regression diffs by kind, latency profile, cleanup ledger. The backing file .vinv/reports/findings.json is the same data, machine-readable — point your agent at it.
7. Regress after any change — exerciser regress <repo> --base-url … replays all banked cases (re-capturing fresh credentials itself) and reports behavior / contract / perf / environment diffs separately, so environment drift never masquerades as a code regression.
8. Hunt waste on demand — "Optimize Latency Hotspots", "Analyze Memory Trends" (Theil–Sen leak suspects), and "Analyze Cache Opportunities" each turn one command into an evidence-seeded fix episode — accepted only if the paired-bootstrap CI clears and behavior stays byte-identical.

Engine CLI reference

Requires identification consolidate first for apis.json, and — for real coverage — a service running under tracelens.

MCP tools reference

vinv-index — your services' code and the session:

vinv-runtime — the captured runs (read-only, provenance-stamped):

vinv-exercise — closes the endpoint-testing loop: your agent exercises your service and reports the run back, and Vinv grades what came back.

FAQ

Privacy

• Everything on your machine — per-repo state in .vinv/ (auto-gitignored), per-machine in ~/.vinv/. No account, no API keys, no telemetry — none.
• One outbound request, and you can read it: a GET of a static JSON file at notices.vinv.ai on activation, for broken-release and security notices only. No query string, no identifiers, no version, nothing uploaded; at most once per 12 hours; the URL is a constant no setting can repoint; disable with vinv.notices.enabled.
• The only download is the embedding model (Hugging Face, once, ~500 MB); everything else builds from this repo.
• Traces store bounded summaries, not raw values; sensitive parameter names (password, token, api_key, …) are redacted, never captured.
• The only LLM Vinv talks to is the coding-agent CLI you configured, through its own auth.

Contributing & license

See CONTRIBUTING.md — uv sync, cargo build in index/, npm install && npm run check in extension/, keep tests/e2e/planted_bug_golden/run.py green. Good first issues are labeled. By taking part you agree to our Code of Conduct; to report a vulnerability, see SECURITY.md. Apache License 2.0 © 2026 VinvAI.