AgentVision

AgentVision

A debug flight recorder for AI coding agents. It watches one program you aredebugging — screenshots its window on a timer, parses its logs with 658 formatadapters, and time-aligns the two — then exposes the result to an AI agent as 90MCP tools.

Everything runs locally on your own CPU. Capturing, hashing, diffing and parsingare free; the only expensive resource is the agent's context, so AgentVisionspends its own compute to keep that small.

your program ──┬── window screenshots ──┐
               │                        ├── one time-aligned timeline ── 90 av_* MCP tools ── your AI agent
               └── logs (658 adapters) ─┘

Why this exists

An AI agent debugging a running program is working blind. It can read your sourceand it can run commands, but it cannot see the window, and it only learns what alog says if it thinks to go and read the right file at the right moment.

AgentVision gives it both halves at once, on one timeline: what the programprinted, and what the screen looked like when it printed it.

The hard part is not collecting that. It is handing it over without corrupting it.Long contexts measurably degrade — every frontier model loses accuracy as inputgrows, well before its window fills, and a model that takes a wrong turn in amulti-turn conversation tends not to recover. A debug tool that invents one detailis worse than no tool, because the agent commits to it. So the governing rulehere is:

AgentVision may not assert anything it did not verify.

That rule is not aspirational. It is enforced in code and in 54 test suites, mostof which exist because a specific version of this tool once said "healthy" whilethe program was failing.

What that looks like in practice

• A summary never replaces evidence. Collapsing 21,982 identical log lines toline [x21982] is allowed — nothing is lost. Dropping, re-levelling or ranking adistinct line is not. Raw output is always available and is never withheld.
• Silence is distinguished from absence. "OCR read the screen and found noerror" and "OCR could not read the screen" are different facts, trackedseparately, because conflating them once made a frame showing an error dialogthe first thing deleted.
• Machine-read values are corroborated, not quoted. OCR misreads0x5D80000 as 0x5080000 and reports full confidence while doing it, soscreen-read hex, ids and counts are checked against the time-aligned log andmarked appears_in_log: true|false — never verified.
• Retractions are as loud as claims. If AgentVision told your agent theprogram died and it is now running, it says so, quotes its own earlier words,and states a counted observation — and stays silent when it merely stoppedlooking.
• Truncation is always reported. A scan that did not finish says so, because"signal absent" and "file never opened" must not look the same.

The part that surprises people

AgentVision does not decide what to install into your program. Your agent does,once. On first connection it refuses to guess and returns a catalog: everyemitter, adapter and tool it could use, alongside real evidence scanned from yourcode (71 signals — does it read keypresses, own a render loop, swallowexceptions, drive a GPU from Python, run in a browser). The agent commits a plannaming what it chose and why. AgentVision builds exactly that.

It refuses lazy plans: a stale catalog token, a selection with no reason, or"install everything" is rejected. The gate fires once per program, ever.

Install

Requires Python 3.11+.

pip install -r requirements.txt

Then follow the guide for your platform — each covers registering the MCP serverwith Claude Code and granting the OS screen-capture permission, both of whichfail silently if skipped:

• macOS — SETUP.md
• Windows — SETUP-Windows.md
• Linux — dist/linux/ (X11 and Wayland notes, init templates)

Optional: pip install -e . gives you the agentvision command(agentvision doctor, agentvision run -- <cmd>). Without it, usepython3 -m python_backend.cli <subcommand>.

Ships with no saved programs — you add your own.

If you are an AI agent reading this

Call av_start_here(). It reports the target program, whether the bridge isbuilt, and your exact next call. Then readdocs/AI_START_HERE.md anddocs/BRIDGE_PROTOCOL.md — the first-connectioncontract is written for you, including what to do when the active profile is notthe program you were asked about.

Documentation

Tests

python3 run_all_tests.py

54 suites. They are worth reading as documentation of real failures: each onetends to encode a specific way this tool once misled its caller.

One of them, bridge_gate, tests the first-connection contract and so needs alive bridge; it SKIPs (loudly, saying how to run it) if none is listening. Toinclude it:

python3 python_backend/api/bridge_server.py --no-autocapture &
python3 run_all_tests.py

Platform support

The Windows and Linux ports are built and unit-tested but have not been re-runend to end since a recent large refactor. macOS is the verified path.

If you are on Windows and want to help close that gap, start withdocs/WINDOWS_PORT_COMPLETION.md — it listswhat is done, the one highest-risk path to test first, and the traps that alreadycost time so you do not rediscover them.

Licence

MIT — see LICENSE.