Halo

Halo

Ask your AI coding agent "what do I click?" — and a big red circle appears on your realscreen, over the exact UI element. Works with both Claude Code and Codex, becauseHalo ships as a standard MCP server.

The macOS world has this (Clicky, Screen Annotations); Windows didn't — unless you have aCopilot+ NPU. Halo fills that gap for Windows + Claude Code / Codex.

How it works

┌──────────┐  MCP (stdio)  ┌──────────────────┐  HTTP 127.0.0.1:7333  ┌──────────────────┐
│  Agent   │◄─────────────►│  Halo MCP server │──────────────────────►│  Overlay renderer │
│ (Claude/ │  take_screenshot│  (python, stdio) │   POST /circle /clear │  (PySide6, GUI,   │
│  Codex)  │  highlight/clear│                  │                       │  always-on-top)   │
└──────────┘               └──────────────────┘                       └──────────────────┘
                                 captures screen (mss)                    draws the red ring

1. The agent calls take_screenshot — Halo captures your primary display, downscales it tofit the token budget, and hands the agent the image plus the scale it used.
2. The agent looks at the image, decides which pixel you should click, and callshighlight(x, y, label="Click here") with the coordinate as it appears in that image.
3. Halo converts image-space → physical-screen pixels and tells a transparent, click-through,always-on-top overlay to draw a labeled ring there. It fades out after a few seconds.

Two processes because an MCP server dies with each agent session, but a GUI overlay mustpersist and own a Qt event loop. They talk over localhost HTTP + JSON — language-agnostic andtrivially curl-testable.

The hard part: coordinate fidelity

Windows scales UI (125% / 150% / 200%). A screenshot is in physical pixels; a naive Qtwindow draws in logical pixels; a downscaled screenshot is in a third space. Get anyconversion wrong and the circle lands in the wrong place. Halo pins everything to physicalpixels:

• Both processes are Per-Monitor-v2 DPI aware (SetProcessDpiAwarenessContext(-4)).
• Qt's own high-DPI auto-scaling is disabled (QT_ENABLE_HIGHDPI_SCALING=0) so 1 Qt unitequals 1 physical pixel — matching mss.
• take_screenshot returns a scale; highlight multiplies by 1/scale before drawing (the"coordinate handshake"), so a pixel picked on a 1280-wide thumbnail lands correctly on a 4Kdisplay.

Verified exact on a 3840×2160 primary at 200% scaling, from a 1280-px screenshot.

Install

Requires Python 3.11+ on Windows.

cd H:\vscode\halo
python -m venv .venv
.\.venv\Scripts\activate
pip install -r requirements.txt
python -m overlay          # M0 check: a transparent, always-on-top window opens

Register with your agent

Use the absolute path to mcp_server/__main__.py, not -m mcp_server. The VS Codeextension launches MCP servers from the workspace root, where -m mcp_server fails withNo module named mcp_server. The absolute script path is cwd-independent and works everywhere.

Claude Code:

claude mcp add-json -s user halo '{"type":"stdio","command":"H:/vscode/halo/.venv/Scripts/python.exe","args":["H:/vscode/halo/mcp_server/__main__.py"],"cwd":"H:/vscode/halo"}'

Then reload the VS Code window / start a new chat. Verify with claude mcp list → halo ✓ Connected.

Codex — add to ~/.codex/config.toml:

[mcp_servers.halo]
command = "H:/vscode/halo/.venv/Scripts/python.exe"
args = ["H:/vscode/halo/mcp_server/__main__.py"]
cwd = "H:/vscode/halo"

Then start a new Codex session. Templates for both live in config/.

The MCP server auto-launches the overlay the first time it's needed, so there's nothing elseto start.

Or use the skill (Claude Code, no MCP)

If you'd rather not depend on the MCP connection, Halo also ships as a Claude Code skillthat drives the same overlay through a plain CLI (cli.py) over Bash — no MCP tools required.Install it once:

mkdir $HOME\.claude\skills\halo
copy skill\SKILL.md $HOME\.claude\skills\halo\SKILL.md

Then in any Claude Code chat just ask "what do I click?" — the skill runspython cli.py screenshot / highlight X Y for you. The MCP server remains the path for Codex(which has no skill system); the two share the overlay and the coordinate handshake.

Use it

Just ask, in either agent:

"What do I click to change the theme?"

The agent screenshots, reasons, and Halo circles the button on your screen. Follow-ups like"a bit lower" work too — it just calls highlight again.

Tools (MCP)

The overlay also exposes a plain HTTP API (POST /circle /box /arrow /clear, GET /health) on127.0.0.1:7333 — see tests/test_overlay_curl.md.

Standalone overlay (no Python needed)

The overlay can run as a one-file Windows executable, so non-developers don't need a Pythonenv. Grab halo-overlay.exe from the Releasespage and run it — it listens on 127.0.0.1:7333 just like python -m overlay. Drop asettings.json next to the .exe to change hotkeys/colors/token budget.

Build it yourself:

pip install pyinstaller
python build.py          # -> dist/halo-overlay.exe (~48 MB, windowed)

(The MCP server / CLI still run from the agent's Python env; they auto-launch whichever overlaythey find — the .exe is just a dev-env-free alternative. First launch unpacks to temp, so itcan take a few seconds; SmartScreen may warn on an unsigned exe.)

Testing

• Overlay smoke tests: tests/test_overlay_curl.md
• DPI / coordinate fidelity: tests/test_coords.md
• End-to-end MCP round-trip: python tests/mcp_client_test.py

Limitations (v1)

• Primary display only. Multi-monitor is intentionally out of scope for v1.
• Exclusive-fullscreen apps/games can draw over the topmost overlay; use borderless-windowed.Fine for the target use case (installers, web UIs, dialogs, IDEs).
• On Windows the overlay's lifetime is tied to the agent session that launched it; itre-launches automatically each session. Start python -m overlay manually for an always-oninstance.

Stack

Python 3.11+ · PySide6 (overlay) ·mss + Pillow (capture) ·MCP Python SDK (FastMCP, stdio).

See PLAN.md for the full architecture, the coordinate traps, and the build log.