Analysis Gym

Analysis Gym

Analysis Gym is a tiny MCP server for recording and scoring prospective equityearnings predictions made by AI agents.

It deliberately does not choose tickers, schedule runs, or invoke models. Youragent loop owns those decisions. The agent uses the existingFactIQ MCP server for research and calls Analysis Gym onlyto record a prediction, record the eventual actuals, or read the results.

Tools

• record_prediction records an immutable forecast before the expectedearnings time.
• record_actuals settles all earlier predictions for a ticker and fiscalperiod.
• get_results returns per-metric errors and a leaderboard grouped by harness,model, and thinking setting.

The five predicted values are revenue, EBITDA, net profit, free cash flow, andthe first regular-session closing price after the earnings release.

Run locally

uv sync
uv run analysis-gym

The server uses stdio transport and stores data in analysis_gym.sqlite3 in itsworking directory. Set ANALYSIS_GYM_DB_PATH to put the database elsewhere.

Codex

Add the server to ~/.codex/config.toml:

[mcp_servers.analysis-gym]
command = "uv"
args = ["--directory", "/absolute/path/to/analysis-gym", "run", "analysis-gym"]

Install and authenticate the FactIQ plugin separately. Then ask Codex, forexample:

Pick an equity reporting soon. Use FactIQ to forecast its next-quarterrevenue, EBITDA, net profit, free cash flow, and first post-earnings close.Record the forecast in Analysis Gym before the release.

Claude Code

claude mcp add analysis-gym -- \
  uv --directory /absolute/path/to/analysis-gym run analysis-gym

Use the same prompt and ensure the FactIQ plugin is also installed andauthenticated.

Agent-side loop

A loop outside this repository can choose an upcoming event and run the samerequest through any set of CLI/model/thinking configurations. Each agent callsrecord_prediction itself. After earnings, call record_actuals once with asource URL, then use get_results to compare the configurations.

Analysis Gym uses symmetric mean absolute percentage error (SMAPE), where loweris better. It reports every metric separately and a simple mean across all five.

Metric definitions

• EBITDA: operating income plus depreciation and amortization.
• Free cash flow: operating cash flow minus capital expenditure.
• Net profit: consolidated net income attributable to the parent/commonshareholders.
• Post-earnings close: the same session's close for a pre-market release, or thenext regular session's close for an after-hours release.

All four financial values (submitted in millions) in a submission must use the same reporting currency.

Development

uv run pytest