Groundlane

Groundlane is an open-source remote MCP server and trusted content access layer for AI agents. Today it provides one controlled interface for Web search, retrieval, deterministic extraction, and URL/raw-HTML parsing. Its full document-processing roadmap covers structured conversion, deterministic extraction, and explicit model-assisted parsing across Office/ODF/RTF, spreadsheets and structured text, EPUB/email, text/scanned/complex PDFs, images, audio, tables, formulas, figures, metadata, and citations. Document sources will use bounded inline bytes, policy-checked public URLs, or opaque storage-neutral artifact references; the Cloudflare reference flow will let an upload-capable client, Groundlane CLI, or dashboard PUT large/private files directly to an R2 staging object using a short-lived presigned URL, then mint an opaque artifact reference only after verification and immutable finalization. The temporary R2 coordinate is not the public artifact identity and is never reused as a tool argument. Document processing does not implicitly create a durable artifact, corpus membership, or index; the future file/document tool family may use the disclosed bounded transient cache described below. The roadmap also includes an explicit operator-owned corpus control plane: Groundlane will own the portable corpus identity, source enrollment, access, freshness, deletion, and citation contracts while keeping indexing and ranking backends replaceable. It is model-neutral, routes replaceable providers behind stable contracts, and keeps authentication and resource limits under the operator's control. Each roadmap format and corpus capability remains unsupported until its own security, fixture, provenance, isolation, and quality gates pass.

[!IMPORTANT]Groundlane is an early preview (0.1.0). Tool contracts and deployment behavior may change. The target OSS V1 Stable Release is an operator-hosted open-source product; Managed Groundlane Cloud is a later roadmap item, not an available service. Groundlane is not a CAPTCHA solver or a universal anti-bot bypass.

OSS V1 Stable is planned as a Web + document release rather than a Web-only release. Its stable document profiles target text-based PDF; DOCX, XLSX, and PPTX; CSV, TXT, Markdown, JSON, XML, and HTML; plus bounded ODF, RTF, EPUB, and EML profiles with explicit omissions and rejection of active, encrypted, nested, or unsupported content. Scanned-PDF/image OCR, legacy Office conversion, complex layout/table/formula/figure recovery, scholarly extraction, and bounded audio transcription may ship only as opt-in experimental engines until their own quality, isolation, cost, and provenance gates pass. Video processing and universal document fidelity are not V1 promises. None of these file/document capabilities are implemented by the current parse tool yet.

The document roadmap uses configurable, bounded retention rather than silent permanent storage. Working defaults are a 15-minute upload intent, a one-hour staging cleanup window, a 24-hour transient artifact, and a 24-hour ownership-scoped processing cache. Callers may adjust upload, artifact, and cache expiry within operator-advertised bounds; out-of-range requests are rejected instead of silently clamped. The staging cleanup window is operator-controlled. Operators may change defaults/maxima or disable caching through an observable document policy. Explicit corpus enrollment uses its own retention policy and defaults to retention until removal; expiry extension is always explicit.

The authoritative future document output is a versioned, provider-neutral canonical document envelope, serialized as JSON in the first contract. Its reusable content core is cached separately from per-source identity and provenance, preventing equal bytes from different URLs or artifacts from inheriting the wrong source. Markdown is the default agent-facing projection, not the source of truth; text, structured, and all-output modes are explicit options. Projections declare their version, lossiness, omissions, and canonical source references. Any oversized document output returns a typed, opaque result ArtifactRef; it is distinct from a cache entry and cannot be reused as a source without an explicit conversion. The existing URL/raw-HTML parse schema remains compatible and does not yet implement this roadmap contract.

Future document execution uses an explicit dual track. Bounded deterministic conversion and extraction may complete synchronously within one end-to-end deadline; long-running work uses a caller-created async job. Both tracks share the same canonical envelope, artifact, cache, provenance, policy, and error contracts. A synchronous request is never silently converted into a durable job because of timeout, output size, queue pressure, fallback, or engine choice. The first async slice will proxy provider-owned document jobs after MCP Tasks compatibility testing; explicit start/status/result/cancel tools are the fallback for clients without adequate task support. Groundlane-owned long-running OCR, layout/VLM, audio, or other document execution requires a separate durable-orchestration, volume, cost, and isolation gate. This lifecycle is roadmap work and is not implemented by the current parse tool.

Tools at a glance

Fetch/extract/parse results report retrieval provenance such as engine, backend, finalUrl, bytes, and truncated when they fetch a URL. Automatic search defaults to batches of at most two complementary providers, canonical-URL deduplication, and RRF while retaining selected/attempted/succeeded provider provenance; if a federated batch has no successful provider, Groundlane tries the next eligible batch within the same deadline. Non-explicit web_search fallback treats a single provider rejection, timeout, quota error, 5xx, or malformed response as a warning and continues to the next eligible provider; an explicit provider preserves that provider's error instead of silently switching sources. Provider-backed tools such as web_answer, web_research, web_content, web_map, web_crawl, web_news, and web_images default to parallel fan-out and return each provider result separately instead of synthesizing them. Pinning a provider stays single-source. web_fetch, web_extract, and URL-backed parse work without a search-provider key.

Provider vendors expose more APIs than Groundlane currently wires into MCP. See provider inventory for the verified feature backlog and the distinction between vendor capability, implemented Groundlane tool, live smoke, account balance evidence, and Groundlane's local attempt budgets.

Use provider_quota as the first diagnostic view when a provider-backed tool exhausts local attempts or web_search returns zero results: it shows provider account-balance status, Groundlane's local provider-dispatch budgets, implemented tools, and searchRouting hints together. Use provider_balance for provider-owned account credits only, and search_budget_status when you specifically need the raw local attempt counters. A balance result of not_configured means the runtime lacks the credential needed for that provider's balance API, not that keyless quota is exhausted.

Research compatibility

web_research deliberately keeps one synchronous MCP contract even when an upstream provider is asynchronous. You.com Research and Parallel Responses return synchronously. Linkup Research creates an upstream task with POST /v1/research, then Groundlane polls GET /v1/research/{id} inside the same request deadline and returns the completed report when available.

Long Linkup research jobs can outlive the MCP request. In that case Groundlane returns a bounded timeout/cancellation error instead of blocking indefinitely; the upstream provider task may still continue outside Groundlane. Use effort=lite, strategy=fallback, and provider=linkup when you want the cheapest bounded Linkup path.

Quick start

Requirements: Node.js 22+, pnpm 10, and Git. Chromium is needed only when the local browser backend is enabled.

git clone https://github.com/vincentxuu/groundlane.git
cd groundlane
pnpm install
pnpm exec playwright install chromium
cp .env.example .env

Set a long random GROUNDLANE_AUTH_TOKEN in .env, then start the server:

set -a
source .env
set +a
pnpm dev

Groundlane now exposes an authenticated Streamable HTTP MCP endpoint at http://localhost:8080/mcp. Search keys are optional; add them only for the providers you want to enable. Keenable can run without a key through its public endpoint, and You.com can run without a key through its free MCP Search profile; set provider keys only when you want authenticated account allowances.

Deploy to Cloudflare

For a fresh Cloudflare deployment, authenticate Wrangler, create the OAuth KVnamespace, inspect the target's configured secret names, enter the tworequired authentication secrets and any optional provider keys, then deploy:

pnpm exec wrangler login
pnpm exec wrangler whoami
pnpm exec wrangler kv namespace create OAUTH_KV
# paste the returned id into wrangler.jsonc's kv_namespaces[0].id
pnpm secrets:status
pnpm secrets:setup
pnpm run deploy

These secret commands affect Cloudflare only; they do not read or update thelocal .env. Without --env, Wrangler uses the top-level target inwrangler.jsonc; if you select a named environment, use that same --env forstatus, setup, and deploy.

Two authentication secrets are required, and must be different values:

• GROUNDLANE_AUTH_TOKEN — the bearer token headless/CLI clients (Codex,Claude Code, scheduled cloud automation) send to /mcp.
• OAUTH_OWNER_PASSPHRASE — gates the /authorize consent screen shown tointeractive cloud connectors (claude.ai, ChatGPT). Reusing the bearer tokenhere would let a phished consent page leak the same credential everyheadless client uses, so generate it separately.

Generate each with at least 32 random characters, for example:

openssl rand -hex 32

Save both in a password manager. Run pnpm secrets:setup; at the numberedprompt these two are listed under authentication (GROUNDLANE_AUTH_TOKEN,then OAUTH_OWNER_PASSPHRASE) — select both, e.g. 1,2, then paste eachvalue when prompted (input is hidden, nothing is echoed back). Search-providerkeys are optional. Use pnpm secrets:setup -- --help to inspect the safeinteractive flow. Setup first presents one numbered list: select multiplesecrets with an entry such as 2,4-6, then it prompts only for those valuesand sends one bulk update. To paste everything once, copycloudflare-secrets.example.env to the ignored.cloudflare-secrets.env, fill the values you use, then run:

pnpm secrets:setup -- --from-file .cloudflare-secrets.env --dry-run
pnpm secrets:setup -- --from-file .cloudflare-secrets.env

The import accepts .env or JSON, rejects unknown names, and never printsvalues. Delete the populated file after setup if you do not need it locally.Then follow the Cloudflare deployment guideto verify health, readiness, authentication, and MCP behavior.

Pushes to main automatically deploy after the CI quality job succeeds. Therepository must have CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN GitHubActions secrets, plus GROUNDLANE_AUTH_TOKEN for post-deploy smoke; seeContinuous deployment.

Cloudflare Container deploys build a Docker image locally before upload. If pnpm run deploy stalls while loading Docker Hub metadata or pulling node:22-bookworm-slim, check the local Docker credential helper first; this is a local Docker/registry problem, not a Worker or TypeScript build result. The production smoke test remains the final deployment proof:

GROUNDLANE_MCP_URL="https://your-worker.example/mcp" pnpm smoke

CI runs pnpm run wait:container and pnpm run smoke:retry after deploy, so asuccessful run means the Cloudflare Container application has left provisioningand the deployed MCP server responds with the expected tool contracts. Atruntime, the Worker also starts the named Container instance beforeauthenticated /readyz and /mcp requests when Cloudflare reports it as notrunning.

Connect an MCP client

Export the same token in the shell that starts your client:

export GROUNDLANE_AUTH_TOKEN="your-long-random-secret"

Codex

codex mcp add groundlane \
  --url http://localhost:8080/mcp \
  --bearer-token-env-var GROUNDLANE_AUTH_TOKEN

Claude Code

claude mcp add --transport http --scope user groundlane \
  http://localhost:8080/mcp \
  --header "Authorization: Bearer ${GROUNDLANE_AUTH_TOKEN}"

The Claude Code command expands the token into its MCP configuration. For shared or production machines, use a secret-backed header helper instead of storing a plaintext token.

These bearer-token steps also cover headless and scheduled cloud automation(cron jobs, cloud routines, workflow runners): configureGROUNDLANE_AUTH_TOKEN as a secret in that platform once, no OAuth needed.

claude.ai / ChatGPT (OAuth)

Interactive cloud connectors expect OAuth, not a pasted API key. Addgroundlane as a custom connector using your deployed Worker's /mcp URL(https://your-worker.example/mcp). Modern clients can register through CIMDwithout a separate pre-registration step; the DCR compatibility endpoint(/register) is bearer-protected to avoid unauthenticated OAuth state growth.See Cloudflare deployment for the exact flow.The connector opens a consent screen after registration. Enter theOAUTH_OWNER_PASSPHRASE you configured during deployment to approve — this isa separate secret from GROUNDLANE_AUTH_TOKEN, used only to gate that consentscreen.

Make the first call

Ask the client to call web_fetch with:

{
  "url": "https://example.com/",
  "format": "markdown",
  "render": "never"
}

The structured response includes an envelope like this (abridged):

{
  "ok": true,
  "data": {
    "finalUrl": "https://example.com/",
    "title": "Example Domain",
    "content": "This domain is for use in illustrative examples...",
    "engine": "http",
    "backend": "direct",
    "truncated": false
  }
}

Use pnpm smoke while the server is running to verify the MCP handshake plus web_fetch and web_extract against example.com.

Why Groundlane?

• One MCP contract: clients do not need provider-specific tool schemas.
• HTTP first: ordinary reads avoid browser cost; Chromium is reserved for rendering and wait conditions.
• Deterministic extraction: CSS selectors produce structured output without an unrequested model inference step.
• Bounded by default: URL policy, DNS/redirect checks, one deadline, byte/output caps, and concurrency limits remain in the Groundlane boundary.
• Explicit hosted fallbacks: Jina Reader and Browserless receive a preflight-validated public final URL only when the operator enables them.

Run Groundlane

Supported adapters

Provider capabilities, pricing, and free allowances

Verified against public official pricing and billing pages on 2026-08-30. Prices below are public USD list prices before applicable tax; enterprise contracts and logged-in account offers may differ. “Groundlane tools” lists implemented runtime paths, not every product the vendor sells. Free monthly/daily allowances, balance top-ups, ongoing rate-limited access, and one-time signup credits are deliberately kept distinct. See the detailed free search and scraping comparison for the accounting method and broader browser/scraping market.

Provider-backed routing can apply conservative per-instance monthly and daily attempt budgets. These are safeguards, not provider billing truth; provider dashboards and spend limits remain authoritative. provider_balance reports account balances only for providers with implemented official balance APIs, currently Linkup, You.com, Firecrawl, and SerpApi. Exa, Browserbase, and Cloudflare are better modeled as usage/cost diagnostics. See Configuration for credentials, routing, limits, and budget semantics, and Provider inventory for the current production provider status, capability matrix, and balance API verification.

Provider selection

Automatic web_search uses the configured SEARCH_PROVIDER_ORDER, capability filtering, provider health, and attempt budgets. The default order favors renewable or account-backed providers first, keeps keyless Keenable and You.com available as low-friction fallback paths, and keeps finite-trial providers opt-in when their free allowance is not renewable or not measurable through an API. Explicit provider calls bypass automatic selection but still require credentials, capability support, URL safety checks, and configured budgets.

For provider-backed tools other than web_search, strategy=parallel returns attributed results from every selected provider; strategy=fallback stops at the first successful provider to reduce spend.

Runtime and billing boundaries

Cloudflare is Groundlane's production runtime today, and it also exposes adjacent capabilities that could become future Groundlane adapters. AI Search is a managed search service for operator-provided data with Workers, REST, and MCP access. Browser Run / Browser Rendering exposes content, markdown, screenshot, PDF, accessibility tree, links, crawl, and structured JSON browser actions through REST APIs or Workers bindings. Agents and Workflows provide durable agent sessions, scheduled work, WebSockets, recoverable steps, and tool orchestration. AI Gateway can add model observability, caching, retries, rate limiting, and fallback.

Those services are not the same thing as the public-web search providers in the provider router. Cloudflare is therefore not listed under provider_balance: that tool is reserved for web-data provider account balances exposed by official provider APIs, currently Linkup credits, You.com API credits, Firecrawl remaining credits, and SerpApi searches left.

Cloudflare usage must be tracked through the Cloudflare dashboard, billing exports, logs, metrics, or future Cloudflare-specific diagnostics. Container cost is based on active runtime resources such as vCPU, memory, disk, egress, Workers, Durable Objects, and logs; those units are separate from search-provider requests or credits. Groundlane local budgets do not cap Cloudflare runtime spend.

Potential Cloudflare-specific Groundlane work should stay separate from search-provider routing: a Browser Run backend for rendered web_fetch / web_content, an AI Search adapter for private/operator-owned indexes, Cloudflare diagnostics for runtime usage, and Workflows-based async tools for long research or crawl jobs.

Large generated documentation sites need source-aware parsing instead of raw page extraction. Cloudflare's docs publish Markdown pages, scoped llms.txt / llms-full.txt indexes, and OpenAPI schemas for the API reference. Groundlane should prefer those machine-readable sources for Cloudflare docs and other similar sites, then slice by product, endpoint, heading, or schema operation. Raising maxBytes or selecting the whole main element is a last resort because it can exceed output limits before the useful section is isolated. The current runtime path proactively handles likely documentation URLs for Markdown/text web_fetch requests by trying the same URL with Accept: text/markdown, trying Cloudflare-style /index.md, then checking same-origin scoped/root llms.txt manifests for a nearest Markdown page after bounded direct failures. Generic machine API paths such as /api/v1/... are not treated as documentation solely because they contain an api segment, and source discovery never suppresses request deadlines or cancellation. Source Markdown cleanup removes front matter and common docs chrome before normal output truncation. OpenAPI slicing exists as pure JSON logic and is not automatically wired into runtime fetch until large schema discovery is bounded.

How it works

MCP client
    |
    v
Worker / Node HTTP edge       authentication, request identity
                              Cloudflare hosts this layer in production
    |
    v
tool registry                 web_search | web_answer | web_research | web_content | web_map | web_crawl
                              web_news | web_images | web_fetch | web_extract | parse
                              diagnostics: provider_quota | provider_balance | search_budget_status | provider_capabilities
    |
    +-- provider router       replaceable search adapters
    +-- safe HTTP + Reader    bounded retrieval and readable content
    `-- browser backend       isolated local or hosted rendering

Core policies do not depend on a search provider or browser runtime. Groundlane Reader uses Mozilla Readability with a local fallback for selector-free Markdown/text; raw HTML and explicit selectors retain deterministic DOM semantics. See Architecture and the reproducible Reader benchmark.

Security and limitations

Web retrieval is SSRF-sensitive. Groundlane treats user URLs, redirects, provider-returned URLs, browser subresources, WebSockets, and DNS answers as untrusted. Keep authentication enabled, preserve the default limits, and apply an outbound network policy in production.

Groundlane does not guarantee CAPTCHA solving, invisible automation, or access to content the operator is not authorized to retrieve. Rendering JavaScript is not proof of anti-bot bypass. The local browser gives a detected access challenge at most five seconds to clear; if the original request deadline has not expired first, a persistent challenge returns retryable UPSTREAM_ERROR at browser-challenge. web_fetch does not automatically spend provider credits by switching to web_content; callers must opt into provider-backed retrieval explicitly. See SECURITY.md for the threat model and private vulnerability reporting.

Project status

• Current source version: 0.1.0 early preview; no stable tool-contract guarantee yet.
• Implemented: ten web access MCP tools, one parser MCP tool, four provider diagnostic MCP tools, thirteen search adapters, provider-backed answer/research/content/map/crawl/news/images paths, self-hosted Reader, optional Jina/Browserless backends, and Cloudflare Worker + Container deployment.
• Next: add a single-tenant multi-credential principal contract, a D1-backed managed-token registry, and an operator CLI backed by an admin-only API. A new admin secret will be isolated from the existing GROUNDLANE_AUTH_TOKEN, which remains a legacy/local data-plane credential and never gains credential-management privileges. Other next steps are hardening tool contracts and compatibility fixtures; preserving machine-readable Reader/parser/extractor benchmark artifacts; designing provider-neutral durable crawl create/status/result/cancel; evaluating single-URL, caller-schema provider extraction; adding stateless login/challenge diagnostics; and running an MCP Tasks/target-client compatibility spike before choosing an async research API. Short research remains synchronous and provider results remain separated. The approved document-source contract is bounded inline bytes, policy-checked public URLs, or Groundlane-issued opaque ArtifactRef; the Cloudflare reference upload path uses an MCP-created provisional upload intent, an upload-capable client/CLI/dashboard, direct presigned PUT to an R2 staging object, and verification/immutable finalization before minting the artifact reference. Self-hosted deployments may replace the artifact backend. Operator-owned corpus search is an accepted product direction, but it will use a separate, demand-gated corpus_search tool family with explicit corpus, freshness, access-control, retention, deletion, and backend provenance rather than changing public-Web web_search. Generic LLM extraction, monitoring/scheduling, persistent authenticated browser sessions, and Groundlane-owned durable orchestration remain demand-gated roadmap items rather than committed runtime features. Any future authenticated-browser slice will use a separate opt-in tool family, human login/MFA, provider-owned opaque profile references, explicit owner/TTL/delete controls, and read-only bounded navigation before Groundlane considers credential custody or general account actions.
• Open-source references are split into primary references and watchlist/discovery sources in the product requirements so low-maintenance candidates do not become runtime priorities by default.
• Planned document processing includes an ownership-scoped, content-addressed result cache with a 24-hour working default, bounded caller TTL/cache controls, complete engine/version/source provenance, and deletion/invalidation rules. This does not enable a Phase 0 Web response cache.
• Planned file/document output uses a canonical structured envelope with stable block/source references and typed tables, assets, formulas, citations, capability states, spans, warnings, errors, and engine/model provenance. Markdown remains the default lossy projection; provider raw JSON is never the public contract, and the current HTML parse schema remains unchanged.
• Commercial roadmap: OSS V1 Stable remains an operator-hosted open-source product. Self-hosting never requires a Groundlane Cloud account, license server, activation check, or mandatory phone-home. Managed Groundlane Cloud is an approved later roadmap phase released progressively as Internal Alpha, Invite-only Beta, then Managed Cloud Public Launch. A public no-card trial waits for verified tenant/secret isolation, allowance hard stops, abuse controls, Claude/Codex/Cursor compatibility, provider cost attribution, token revocation, project deletion, and basic incident handling. Cloud uses a hosted Remote MCP endpoint plus Web dashboard, preset-first routing with full provenance, and no silent funding switch. Importing OSS configuration into Cloud remains optional.

The detailed product requirements, capability matrix, roadmap, and acceptance criteria live in the product requirements.

Documentation

• Configuration
• Architecture
• Cloudflare deployment
• Open-source foundations
• Reader benchmark
• Parser benchmark
• Research archive

Contributing and support

Use GitHub Issues for bugs and feature proposals. Read CONTRIBUTING.md and the Code of Conduct before opening a pull request. Report security vulnerabilities privately as described in SECURITY.md.

License

Groundlane is licensed under the Apache License 2.0.