databricks_ops_mcp

A governed MCP server exposing Databricks operations as tools — jobs orchestration,SQL execution, notebook creation, Unity Catalog governance, lineage, clusters, andDLT pipelines. Built as a reuse template: same pattern as a SharePoint/Jira MCPserver, pointed at the Databricks REST API.

Tools (17)

Category	Read-only	Write-gated
Jobs	list_jobs, get_run_status, list_failed_runs	run_job, repair_run
SQL	preview_table	execute_sql (DML/DDL only; SELECT is free)
Workspace	export_notebook, list_workspace	create_notebook
Unity Catalog	list_tables, get_table_grants, get_table_lineage	—
Compute/DLT	list_clusters, list_pipelines	restart_cluster, start_pipeline_update

Safety model

Dry-run by default: every write tool returns a validated dry-run message untilDBX_WRITE_ENABLED=true. Promote per environment (DEV=true, PROD=false + approval flow).
Audit log: every tool call appends a JSONL record (timestamp, tool, payload).
Context safety: results capped at DBX_MAX_RESULT_ROWS / DBX_MAX_RESULT_CHARS.
Read-only SQL enforcement: statements must start with SELECT/SHOW/DESCRIBE/EXPLAIN/WITHunless writes are enabled; multi-statement submissions are rejected.
Placeholders only: no real hostnames, principals, or credentials anywhere in code.Everything comes from environment variables.

Setup

pip install -r requirements.txt
cp .env.example .env      # fill in host, token (PAT or OAuth M2M), warehouse ID

Run

# Local (stdio) — for Claude Desktop / Claude Code
python server.py

# Remote (streamable HTTP) — for Databricks Apps / ECS / Lambda-backed deployments
MCP_TRANSPORT=http python server.py

Claude Desktop config

{
  "mcpServers": {
    "databricks_ops": {
      "command": "python",
      "args": ["/path/to/databricks_ops_mcp/server.py"],
      "env": {
        "DATABRICKS_HOST": "https://workspace_placeholder.cloud.databricks.com",
        "DATABRICKS_TOKEN": "token_placeholder",
        "DATABRICKS_WAREHOUSE_ID": "warehouse_id_placeholder",
        "DBX_WRITE_ENABLED": "false"
      }
    }
  }
}

Claude Code

claude mcp add databricks_ops -- python /path/to/databricks_ops_mcp/server.py

Bedrock / custom agents

Run with MCP_TRANSPORT=http behind your FastAPI gateway or as a Databricks App;point the agent's MCP client at the streamable HTTP endpoint. For Databricks Appshosting, replace the bearer token with app OAuth (on-behalf-of user) so UnityCatalog enforces per-user permissions.

Example flows

"List failed runs from today, get the status of the worst one, and repair it"→ list_failed_runs → get_run_status → repair_run (approval-gated in PROD)
"Create a notebook that deduplicates the claims feed and preview the target table"→ create_notebook → preview_table
"What feeds this table and who can modify it?"→ get_table_lineage + get_table_grants (Conversational Data Steward core)

Extension roadmap

create_job_from_spec — NL → metadata-driven pipeline config → Jobs API
Column-level lineage + cross-platform trace into MSTR cubes (second MCP server)
deploy_bundle via Asset Bundles for Git-first production deployment
Approval elicitation (ctx.elicit) on write tools instead of a global flag
Cost tools over /api/2.0/usage (billing usage export)

Project layout

databricks_ops_mcp/
├── server.py            # FastMCP/MCPServer registration (SDK 1.x & 2.x compatible)
├── client.py            # Shared async REST client, errors, audit, truncation
├── config.py            # Env-driven settings and guardrails
├── tools/
│   ├── jobs.py          # /api/2.2/jobs/*
│   ├── sql.py           # /api/2.0/sql/statements (submit → poll → fetch)
│   ├── workspace.py     # /api/2.0/workspace/* (notebook import/export)
│   ├── unity_catalog.py # /api/2.1/unity-catalog/* + lineage-tracking
│   └── compute.py       # /api/2.1/clusters/*, /api/2.0/pipelines/*
├── requirements.txt
└── .env.example

databricks_ops_mcp