flameox

flameox

Runtime evidence for coding agents

Let an agent query, compare, and audit profiler traces, benchmarks, memory captures, and execution evidence without uploading your code or data.

Quick start  ·  What flameox investigates  ·  How it works  ·  CLI and MCP  ·  Documentation

Connect your agent: npx flameox@latest setup

flameox helps coding agents investigate performance, memory, execution,concurrency, and reliability with evidence you can inspect and reproduce. Itconnects agents to maintained tools such as Python import-time tracing, pytest,xdist, pyperf, py-spy, Perfetto Trace Processor, coverage.py, Memray, andtorch.profiler, then keeps each original artifact alongside a record of how itwas produced.

flameox is not a profiler or an automatic bug finder. It coordinates existingtools, compares runs collected under compatible conditions, and ties findingsback to the measurements that support them.

Quick start

Connect an agent

Run the guided setup:

npx flameox@latest setup

The wizard detects Claude Code, Cursor, OpenCode, Codex, Gemini CLI, andAntigravity. It preselects detected clients for connection and previews everyconfiguration file it will change; you can adjust the selection before applyingit. After you approve the plan, it installs and verifies a versioned localruntime and activates the clients you chose.

Restart the configured client, open a project, and ask:

Initialize flameox in this project and show me which profiling capabilities areavailable.

Client setup and project initialization are separate steps. Setup registers theMCP server, while the initialization request creates .diagnostics/ in thecheckout after you confirm that the client opened the intended project. Theserver does not initialize arbitrary launch directories automatically.

flameox can initialize its .diagnostics/ workspace through MCP. An agent cancreate or update a validated named workload in flameox.toml through MCP, thenplan and run it immediately. SeeNamed workloads and capture for an example.

Use the CLI from source

Python 3.12 or newer and uv are required:

uv sync --extra dev --extra python --extra execution --extra memory --extra trace --extra cpu --extra torch
uv run flameox init .
uv run flameox status

What flameox investigates

Profiles show where to investigate; they do not prove why behavior changed orwhether the program remains correct. To confirm a result, use a representativeworkload and declared metric, compare the same source and environment, retain thesamples, and validate the program's output for both the baseline and candidate.

How it works

1. Declare. An agent creates or updates a validated named workload andFlameox binds its current definition to every plan and run.
2. Capture. A maintained profiler or benchmark tool runs while flameox recordsthe tool, command, environment, source revision, limits, and outcome.
3. Preserve. flameox keeps the original artifact and publishes queryableevidence to the project workspace.
4. Analyze. The CLI and MCP server provide focused queries forhotspots, scaling, memory, execution, failures, and comparisons.
5. Record. Findings remain tied to the runs, measurements, validation, andanalysis that support them. Failed attempts remain visible.

Safety boundaries

flameox runs on your machine and does not upload code or captures. It does notmonitor production, provide accounts or synchronization, modify source code,install system tools, or delete artifacts automatically.

Agents can run named workloads declared through the structured workloadconfiguration path. MCP does not provide arbitrary shell or SQL access, deleteevidence, return raw artifact bytes, or launch native viewers. The default agentcapture runs directly as trusted local execution and reports uncontainedcontainment; a project can explicitly require managed containment.

Setup and installation details

Run setup again to connect or disconnect clients, verify that connected clientslaunch the active runtime, update to the npm package's matching version, or rollback to a previously installed version. Always use npx flameox@latest forsetup and upgrades: an unqualified npx flameox invocation can reuse an oldercached bootstrap. Automation can select clients and inspect the plan explicitly:

npx flameox@latest setup --codex --claude --yes
npx flameox@latest setup --all --dry-run --json
npx flameox@latest setup --verify --yes --json
npx flameox@latest upgrade

The npm package installs the matching flameox Python release. MCP clients thenlaunch that installed runtime directly; they do not call npx, uvx, or anetwork-dependent installer at startup. Setup does not initialize a project orcreate .diagnostics/.

Optional Python extras are independent:

• python: pyperf capture and import
• cpu: py-spy capture
• trace: Perfetto Python API; MCP can stage the pinned user-space TraceProcessor when it is missing
• execution: coverage.py
• test: pytest and pytest-xdist evidence capture
• memory: Memray
• torch: PyTorch capture
• all: all runtime integrations

When FlameOx is connected to an agent, the agent does not need to guess packagenames. list_capabilities reports the exact managed providers that are missing;the agent calls start_capability_setup with an idempotency key, then pollsget_capability_setup (or calls cancel_capability_setup when needed).prepare_capabilities remains as a compatibility wrapper. These actions verifytheir result and never run a workload. prepare_workload_dependencies installsonly Python distributions declared by a named workload. Non-privilegeduser-space tools such as Trace Processor are staged automatically; hostexecutables, permissions, and privileged collectors remain explicit limitations.

Local data model

Initialize a project-local workspace:

uv run flameox init .
uv run flameox status

.diagnostics/ stores original artifacts, run and investigation records, Parquetevidence, and a rebuildable DuckDB catalog. Parquet files and generationmanifests are the source of truth; if you delete catalog.duckdb,flameox catalog rebuild reconstructs it.

Identical artifacts are stored once, but flameox retains the source, environment,workload, and measurement details for every run. Hypotheses, trials, comparisons,and findings remain separate so observations do not blur into conclusions.

Named workloads and capture

Repeatable commands live in flameox.toml. Templates accept declared scalarparameters only—there is no shell expansion:

schema_version = 1

[workloads.scan]
argv = ["python", "bench.py", "--implementation", "{implementation}"]
cwd = "."
timeout_seconds = 60

[workloads.scan.parameters]
implementation = ["baseline", "candidate"]

[workloads.scan.oracle]
strength = "cross_treatment_equivalence"
argv = ["python", "validate.py", "--implementation", "{implementation}"]

[experiments.scan_comparison]
workload = "scan"
variants = ["baseline", "candidate"]
design = "randomized_complete_blocks"
blocks = 10
primary_metric = "pyperf.workload"
polarity = "lower_is_better"
estimand = "median_paired_log_ratio"
practical_threshold = 0.05
confidence_level = 0.95
random_seed = 1984

An agent can create this declaration directly through MCP withconfigure_workload. That operation validates the complete project, preservesunrelated workloads and experiments, and makes the workload immediatelyavailable; it never runs the command. The agent then followslist_declared_workflows (no arguments lists workloads) →get_declared_workflow → list_capabilities → plan_capture → execute_capture_plan.Pass kind="experiment" when discovering experiments. A valid manually authoredworkload is also immediately active. There is no separate workload approval orhuman-check step: the canonical definition in flameox.toml is the executionbinding.

The equivalent CLI capture commands are:

uv run flameox workload show scan --json
uv run flameox capture plan pyperf --workload scan \
  --parameters '{"implementation":"baseline"}' --json
uv run flameox capture run pyperf --workload scan \
  --parameters '{"implementation":"baseline"}' --json

Editing a command, environment, parameter domain, timeout, working directory,or oracle changes the workload definition and invalidates existing plans.Execution uses argument arrays instead of shell strings, bounds command output,and records cleanup after timeouts or cancellation. The default agent path isdirect local execution and reports uncontained containment. Linux users canexplicitly require managed Bubblewrap/systemd containment when the strongerdescendant guarantee is needed.

Investigations and experiments

Create an investigation and optionally attach a falsifiable hypothesis beforerunning a predeclared experiment:

uv run flameox investigations create \
  '{"question":"Does the candidate remove reverse-scan overhead?"}' --json
uv run flameox hypotheses record @hypothesis.json --json
uv run flameox experiment plan scan_comparison \
  --investigation <investigation-id> --adapter pyperf --json
uv run flameox experiment run scan_comparison \
  --investigation <investigation-id> --adapter pyperf --json

Before collecting data, flameox saves the declared protocol. It randomizestreatment order within complete blocks and records every attempted trial,including cancellations and failures. The automatic paired comparison runs onlywhen the trial blocks are complete and the measurements, source, environment,and output validation are compatible. Failed trials remain in the evidenceinstead of disappearing from the denominator.

Useful read-only analyses include:

uv run flameox analyze hotspots <run-or-artifact>
uv run flameox analyze scaling <experiment-id>
uv run flameox analyze compare @comparison-request.json
uv run flameox analyze memory <run-or-artifact>
uv run flameox analyze execution <run-or-artifact>
uv run flameox analyze pytorch <run-or-artifact>
uv run flameox analyze failures

These commands do not modify the workspace. Record a result when you want topreserve it with the runs that produced it:

uv run flameox analyze record \
  '{"recipe":"memory","input_id":"<run-id>"}'
uv run flameox analyze record-comparison @comparison-request.json

From a hotspot, inspect its callers, callees, representative stacks, orsurrounding trace window:

uv run flameox stacks callers <run-or-artifact> <frame-id> [--cursor CURSOR]
uv run flameox stacks callees <run-or-artifact> <frame-id>
uv run flameox stacks examples <run-or-artifact> <frame-id>
uv run flameox trace window <artifact-id> --start 0 --end 1000000 [--cursor CURSOR]
uv run flameox open <artifact-id>

flameox open only prints a native viewer plan. Pass --launch separately toopen the viewer; it cannot be combined with --json.

CLI and MCP

Start the stdio server with a fixed project root:

uv run flameox mcp serve --project-root .

Flameox is published to the official MCP Registry asio.github.morluto/flameox. The registry entry uses the maintained PyPIpackage and launches the same stdio server through uvx; the npm packageremains the guided setup bootstrap. Registry-aware clients can discover theserver by that name. The equivalent direct launch for a specific release is:

uvx --from flameox==VERSION flameox mcp serve --project-root .

The project root is the client's working directory. Flameox does not create its.diagnostics/ workspace until the client explicitly calls the initializationworkflow for that project.

Through MCP, agents can configure workloads, plan captures and experiments, query evidence,preview analyses, and record results. Capture and experiment plans areshort-lived and single-use; restarting the server invalidates those plan tokens.Detached capture records persist across a manager restart: retry the sameidempotency_key to reconnect to the original run rather than starting anothercapture. Capability setup uses the same rule; after a retryable failure orcancellation, follow the returned recovery arguments with their freshidempotency_key to start a new attempt.

Inspect the protocol surface with a real stdio client:

uv run flameox mcp inspect --project-root . --json

Integrity and recovery

uv run flameox validate
uv run flameox validate --full
uv run flameox catalog validate
uv run flameox catalog rebuild
uv run flameox catalog compact
uv run flameox recover
uv run flameox gc
uv run flameox gc --apply

Full validation hashes native artifacts and Parquet files. Recovery closes a runonly after its boot, PID, and process-start lease has disappeared. Garbagecollection is a dry run by default; --apply moves eligible objects torecoverable trash instead of deleting them immediately.

Documentation

• Architecture: process model, package boundaries,dependencies, and platform policy
• Storage and evidence: authoritative data,identity, provenance, publication, and schemas
• Investigations and analysis: workloads, experiments,recipes, statistics, and evidence quality
• Adapters and capabilities: profiler integration,compatibility, probing, and adapter policy
• Runtime safety: concurrency, recovery, retention,integrity, security, privacy, and local observability
• CLI and MCP boundaries: agent interfaces and executionboundaries
• Testing: suite ownership, focused lanes, provider setup,and collection-preservation checks
• Contributing: development setup, project contracts,validation, and pull request expectations

Development

See CONTRIBUTING.md for the full development and pull requestworkflow.

uv sync --extra dev
uv run python tools/test.py list
uv run python tools/test.py core
uv run ruff check src tests tools
uv run mypy src tests tools
uv run pytest -q

pytest has no hidden retries and runs the deterministic core, excludingprocess, optional-provider, and performance lanes. Use the testing guidefor focused subsystem commands, provider matrices, and the collection-preservation receipt.