omnia-v

@error-bar/mcp

Community omnia-v
Updated

errorbar MCP server (@error-bar/mcp) — the platform API as tools for Claude, Cursor, and any MCP client

@error-bar/mcp

The errorbar platform as MCP tools — every public API operation (evals, judges/criteria, gates, logs, datasets, aliases, label sets, audit/proving, dedicated endpoints, training) callable from Claude Code, Claude Desktop, Cursor, or any MCP client.

One tool per API operation, 76 tools, generated from the platform's route contracts. Inputs are validated with the same field names and casing the REST API accepts; responses come back exactly as the API returned them (JSON pretty-printed, exports as text), including the API's own error status and message.

Renamed from @omnia-voice/mcp (2026-08-30). The old name is published as a shim that forwards here; update your config when convenient.

Install

npx -y @error-bar/mcp        # or: npm i -g @error-bar/mcp && errorbar-mcp

Needs ERRORBAR_API_KEY (a workspace API key, sk_…, from Settings → API keys). Give the key only the scopes the agent should have — read for a monitoring agent, read + evals:write for one that runs evals — the platform enforces them and each tool's description says which scope it needs.

Claude Code

claude mcp add errorbar -e ERRORBAR_API_KEY=sk_… -- npx -y @error-bar/mcp

Claude Desktop / Cursor / any mcpServers config

{
  "mcpServers": {
    "errorbar": {
      "command": "npx",
      "args": ["-y", "@error-bar/mcp"],
      "env": { "ERRORBAR_API_KEY": "sk_…" }
    }
  }
}

Profiles — hand the agent the tools for one job

Seventy-six tools is the complete contract and the wrong thing to give a model with one question: every description is read before the first call, and a long list makes the pick worse. A profile is the handful a job needs, plus the task-shaped tools that answer the job's question directly.

npx -y @error-bar/mcp --profile monitor      # or ERRORBAR_MCP_PROFILE=monitor
Profile For Tools
setup connect traffic and reach the first verdict get_setup_status, screen_my_traffic, list_logs, get_trace, list_evals, get_eval, list_criterion_templates, list_criteria, create_criterion, get_criterion, run_criterion_alignment, is_my_judge_trustworthy
monitor is quality holding, can the judges be trusted — read-only by construction get_setup_status, is_my_judge_trustworthy, list_criteria, get_criterion, get_criterion_alignment, get_criterion_certificate, list_alerts, get_judge_settings, get_failure_clusters, list_refusals, list_evals, get_eval
release ship or hold: comparison, gate on the interval, repoint the alias can_i_ship, screen_my_traffic, list_evals, get_eval, create_eval, get_eval_gate, get_eval_evidence, compare_evals, list_aliases, upsert_alias, get_criterion_certificate, verify_document
all (default) everything: all 76 operations plus the task tools

--read-only, --no-spend and --only apply on top of a profile.

Task-shaped tools

Three tools answer the questions people actually ask, with the doctrine applied (verdicts from the interval, never the point estimate) and a pointer to the one-per-endpoint tool for the detail:

Tool Question Does
screen_my_traffic would a cheaper (or newer) model hold on my traffic? starts a zero-config screening against the incumbent's stored answers, waits for it, returns each candidate's verdict with its win-rate interval, cost vs today and similarity to production, plus switch/keep. Spends wallet credit.
is_my_judge_trustworthy can I believe this judge? one criterion or all: trust verdict from the TPR/TNR intervals, catches-failures / passes-clean with 95% CIs, κ, grades, drift, and the one action that changes its state. Read-only.
can_i_ship does the evidence let this change through? gates the newest finished run (or eval_id) with your thresholds — defaults: certified-switch noninferiority_margin 0.05 for a criterion run against stored answers, min_win_rate 0.5 for a comparison — and returns ship/hold with every check's required vs actual (the interval's lower bound). Read-only; the same call is the CI step.

Resources and prompts

Resources: errorbar://agent-setup.md and errorbar://llms.txt (fetched live from www.errorbar.ai, so an installed server never rots) and errorbar://after-traffic-flows (the first five steps, each naming the tool that performs it). Prompts: set-up-errorbar, after-traffic-flows, release-check (eval_id optional).

Flags

Flag Effect
--profile NAME (or ERRORBAR_MCP_PROFILE) setup · monitor · release · all (default) — see Profiles
--read-only (or ERRORBAR_MCP_READ_ONLY=1) expose only GET tools (and the read-only task tools)
--no-spend (or ERRORBAR_MCP_NO_SPEND=1) hide tools that start billable work (eval runs, screening, training, dedicated capacity, judge assists)
--only list_evals,get_eval expose just these tools
--base-url URL (or ERRORBAR_BASE_URL) API root, default https://gateway.errorbar.ai

The OMNIA_* spellings of every environment variable are still accepted.

Tools that spend wallet credit say SPENDS MONEY in their description and are not readOnlyHint; MCP clients that confirm before non-read-only calls will ask for those.

Tools

Tool Operation Scope
list_alerts GET /v1/alerts read
list_aliases GET /v1/aliases read
upsert_alias PUT /v1/aliases aliases:write
delete_alias DELETE /v1/aliases/{id} aliases:write
export_audit_log GET /v1/audit/export read
list_audit_tombstones GET /v1/audit/tombstones read
create_audit_tombstone POST /v1/audit/tombstones platform:write
get_audit_verification GET /v1/audit/verify read
list_batches GET /v1/batches read
create_batch POST /v1/batches platform:write · spends
get_batch GET /v1/batches/{id} read
cancel_batch POST /v1/batches/{id}/cancel platform:write
list_criteria GET /v1/criteria read
create_criterion POST /v1/criteria evals:write
suggest_criteria POST /v1/criteria/suggest evals:write · spends
list_criterion_templates GET /v1/criteria/templates read
get_criterion GET /v1/criteria/{id} read
update_criterion PATCH /v1/criteria/{id} evals:write
delete_criterion DELETE /v1/criteria/{id} evals:write
run_criterion_alignment POST /v1/criteria/{id}/align evals:write · spends
get_criterion_alignment GET /v1/criteria/{id}/alignment read
auto_improve_criterion POST /v1/criteria/{id}/auto_improve evals:write · spends
get_criterion_certificate GET /v1/criteria/{id}/certificate read
scan_criterion_suspects POST /v1/criteria/{id}/scan evals:write · spends
decontaminate_texts POST /v1/datasets/decontaminate evals:write
create_dataset_from_logs POST /v1/datasets/from_logs evals:write
list_dedicated_endpoints GET /v1/dedicated read
create_dedicated_endpoint POST /v1/dedicated platform:write · spends
list_dedicated_templates GET /v1/dedicated/templates read
get_dedicated_endpoint GET /v1/dedicated/{id} read
update_dedicated_endpoint PATCH /v1/dedicated/{id} platform:write · spends
delete_dedicated_endpoint DELETE /v1/dedicated/{id} platform:write
list_refusals GET /v1/enforcement/refusals read
list_env_tools GET /v1/env/tools read
register_env_tool POST /v1/env/tools platform:write
delete_env_tool DELETE /v1/env/tools/{id} platform:write
list_evals GET /v1/evals read
create_eval POST /v1/evals evals:write · spends
compare_evals GET /v1/evals/compare read
get_failure_clusters GET /v1/evals/failure_clusters read
get_eval GET /v1/evals/{id} read
delete_eval DELETE /v1/evals/{id} evals:write
cancel_eval POST /v1/evals/{id}/cancel evals:write
get_eval_evidence GET /v1/evals/{id}/evidence read
get_eval_gate GET /v1/evals/{id}/gate read
get_eval_samples GET /v1/evals/{id}/samples read
list_training_files GET /v1/fine_tuning/files read
upload_training_file POST /v1/fine_tuning/files platform:write
list_fine_tuning_jobs GET /v1/fine_tuning/jobs read
create_fine_tuning_job POST /v1/fine_tuning/jobs platform:write · spends
get_fine_tuning_job GET /v1/fine_tuning/jobs/{id} read
cancel_fine_tuning_job DELETE /v1/fine_tuning/jobs/{id} platform:write
get_fine_tuning_bakeoff GET /v1/fine_tuning/jobs/{id}/bakeoff read
start_fine_tuning_bakeoff POST /v1/fine_tuning/jobs/{id}/bakeoff platform:write · spends
list_grpo_runs GET /v1/grpo/runs read
start_grpo_run POST /v1/grpo/runs platform:write · spends
get_grpo_run GET /v1/grpo/runs/{id} read
stop_grpo_run POST /v1/grpo/runs/{id}/stop platform:write
get_grpo_run_weights GET /v1/grpo/runs/{id}/weights read
list_label_sets GET /v1/label_sets read
create_label_set POST /v1/label_sets evals:write
attach_label_set POST /v1/label_sets/{id}/attach evals:write
freeze_label_set POST /v1/label_sets/{id}/freeze evals:write
list_labels GET /v1/labels read
create_label POST /v1/labels evals:write
list_logs GET /v1/logs read
export_logs GET /v1/logs/export read
list_model_versions GET /v1/model_versions read
get_model_version GET /v1/model_versions/{id} read
adopt_model_version POST /v1/model_versions/{id}/adopt platform:write
list_raft_rounds GET /v1/raft/rounds read
get_judge_settings GET /v1/settings/judge read
set_judge_settings PUT /v1/settings/judge platform:write
get_setup_status GET /v1/setup/status read
get_trace GET /v1/traces/{traceId} read
verify_document POST /v1/verify read

Each tool's description carries the full contract: every query and body field, what the response contains, and the gotchas (feature flags, 400 conditions, cursor semantics).

Programmatic use

import { createServer, ErrorbarClient } from "@error-bar/mcp";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";

const server = createServer({ client: new ErrorbarClient({ apiKey: process.env.ERRORBAR_API_KEY! }), readOnly: true });
await server.connect(new StdioServerTransport());

Development

npm ci && npm run typecheck && npm test && npm run build

src/manifest.ts is generated from the platform's app/api/v1 routes; test/manifest.test.ts pins the exact route inventory it must cover, so a new platform route fails this suite until the manifest is regenerated.

Release

Tag vX.Y.Z here, then run the publish-mcp workflow in omnia-tracing with that tag (dir: . publishes @error-bar/mcp; run it again with dir: legacy to publish the @omnia-voice/mcp forwarding shim) — the npm token lives there.

License: Apache-2.0

MCP Server · Populars

MCP Server · New