Modal Sandbox MCP

Modal Sandbox MCP

Give your AI agent on-demand cloud compute — CPU, RAM, and even GPU — through aModel Context Protocol (MCP) server backed byModal sandboxes.

run_script executes arbitrary Python on Modal's serverless infrastructure withper-second billing, so your agent can crunch through heavy jobs that would crawl(or OOM) on a small VPS — without you renting an always-on GPU box.

Works with Open WebUI (native MCP or via the mcpo OpenAPI bridge), Claude, andany other MCP-capable client.

Why

Your Open WebUI / agent host (e.g. a cheap VPS) usually has:

• limited CPU and RAM
• no GPU
• no burst capacity

Modal provides:

• containers with up to 64+ cores, 100+ GB RAM
• GPUs (T4, A10G, A100, ...) on demand
• scale-to-zero: you pay nothing while idle, only per-second while a script runs
• a free monthly credit (~$30/mo) — plenty for light/medium usage

This server is the glue: a thin MCP wrapper that turns "run this script" into aModal sandbox with whatever specs the agent asks for.

Architecture

┌────────────┐    MCP (stdio or HTTP)    ┌──────────────────┐
│  Client    │ ────────────────────────▶ │  FastMCP server  │
│ (Open WebUI│                           │  (this repo)     │
│  / Claude) │ ◀──────────────────────── │                  │
└────────────┘                           └────────┬─────────┘
                                                  │ modal.Sandbox.create(
                                                  │   cpu=..., memory=...,
                                                  │   gpu=..., timeout=...)
                                                  ▼
                                        ┌──────────────────┐
                                        │  Modal cloud     │
                                        │  (ephemeral      │
                                        │   sandbox)       │
                                        └──────────────────┘

Two deployment flavors are included:

The heavy lifting always happens on Modal either way — the wrapper is just glue.

Setup

1. Prerequisites

• A Modal account (free tier: no payment method required)
• Python 3.10+ (for the local flavor)

2. Install & authenticate

pip install modal fastmcp uvicorn   # or: uv pip install ...
modal token new                     # opens browser; stores creds in ~/.modal.toml

3a. Deploy on Modal (hosted flavor)

modal deploy sandbox_mcp.py

Note the printed URL — it serves MCP over streamable HTTP at <url>/mcp.

3b. Run locally (glue flavor)

python sandbox_mcp_local.py --http --host 127.0.0.1 --port 8020
# stdio mode (for mcpo / MCP stdio clients):
python sandbox_mcp_local.py

Recommended for Open WebUI: wrap it with mcpo so it appears as an OpenAPItool server, exactly like the official Open WebUI MCP servers:

uvx mcpo --port 8021 --name sandbox-mcp \
  --description "Run Python scripts on Modal hardware via Modal Sandboxes." \
  -- /path/to/venv/bin/python /path/to/sandbox_mcp_local.py

A ready-made systemd user unit is in deploy/sandbox-mcpo.service.

Connecting Open WebUI

Via OpenAPI tool server (mcpo):Admin Panel → Settings → Connections → OpenAPI Tool Servers → add:

http://<host>:8021/openapi.json

Then in a chat: + → Tools → enable sandbox-mcp.

Via native MCP:Admin Panel → Settings → Connections → MCP Servers → add:

http://<host>:8020/mcp     (streamable HTTP)

💡 Open WebUI's MCP connection test runs from your browser — a server bound toa private address (e.g. Docker bridge 172.17.0.1) will fail the browser-sidecheck even though the backend can reach it. The OpenAPI/mcpo route is fetchedserver-side and is the reliable choice for containerized Open WebUI.

The tool: run_script

Returns exit code, stdout, and stderr (truncated at 100 KB). Each run is a fresh,isolated, ephemeral sandbox — no persistent state, no access to your host's files.

Sandboxes have network access and come with Python + requests; scripts canpip install extra packages at runtime (adds a little time).

The tool: list_hardware

list_hardware() has no arguments and returns the static hardware catalog — CPU/RAMtiers, GPU types (T4/L4/A10G/L40S/A100/H100), approximate hourly prices, and guidanceon when to use each. Call it first when you need to choose cpu / memory_mb / gpuvalues deliberately instead of guessing. Prices are approximate; verify atmodal.com/pricing.

Tuning timeouts (Open WebUI gotcha)

Open WebUI caps tool-server calls with the aiohttp client timeout. If your scriptsrun longer than 5 minutes, set this env var on the open-webui container(default fallback is only 300s):

AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER=900

(Requires recreating the container; keep it above your sandbox timeout.)

Full stack for reference: Open WebUI 900s → mcpo 900s → sandbox timeout (thereal backstop).

Cost notes

• CPU sandbox runs cost fractions of a cent — effectively free inside Modal'smonthly credit.
• GPUs are the expensive option (~$0.60+/hr). The agent should default to CPU andonly request a GPU when the workload genuinely needs one (see the example systemprompt).
• Scale-to-zero means an idle server costs nothing.

Security notes

• The run_script tool is arbitrary code execution by design. It's for youragent, not strangers.
• Hosted flavor: the Modal URL is unauthenticated — anyone with it can run codeon your Modal account. Add auth (e.g. a bearer-token middleware) before exposingit publicly.
• Local flavor: bind to 127.0.0.1 or the Docker bridge gateway(172.17.0.1) so only your container/host can reach it.
• Modal tokens stay in ~/.modal.toml (or MODAL_TOKEN_ID / MODAL_TOKEN_SECRETenv vars) — never commit them.

Example agent system prompt

See examples/system-prompt.md — copy it into yourmodel's system prompt (Open WebUI: Workspace → Models → edit → System Prompt) toteach the agent when and how to use the tool.

Files

sandbox_mcp.py          # Modal-hosted flavor (modal deploy)
sandbox_mcp_local.py    # local glue flavor (HTTP or stdio/mcpo)
deploy/
  sandbox-mcpo.service  # systemd user unit for mcpo wrapping
examples/
  system-prompt.md      # ready-to-paste agent instructions

License

MIT — see LICENSE.