Nimos-Websites

Website Content Crawler MCP Server

Community Nimos-Websites
Updated

MCP wrapper for the website-content-crawler actor — crawl results as JSON.

Website Content Crawler MCP Server

MCP wrapper that runs the apify/website-content-crawler actor for real via apify-clientand returns its pages as JSON. Every call either returns crawled content or a structurederror — it never reports a fake success.

What it costs

Runs bill your Apify account at the rate listed forapify/website-content-crawler — thisserver adds nothing on top. No token, no charge: calls return missing_token before anyrun starts.

Live output demo (same crawl logic, rendered): https://website-content-crawler.vercel.app

Setup

npm install
export APIFY_TOKEN=apify_api_...   # https://console.apify.com/settings/integrations
npm start                          # stdio MCP server

Tool: crawl_website

Input:

field type default notes
url string required http/https URL to crawl
maxPages number 10 capped at 50
crawlerType string cheerio or playwright:chrome for JS-rendered

Output envelope (same shape every call):

{
  "status": "ok",
  "run": { "id": "<apify run id>", "status": "SUCCEEDED" },
  "page_count": 3,
  "total_in_dataset": 3,
  "pages": [{ "url": "...", "title": "...", "text": "...(≤5000 chars)" }]
}

Error codes: invalid_url, missing_token, apify_auth_failed, actor_run_failed,run_not_succeeded, dataset_fetch_failed.

Smoke test

printf '%s\n' \
 '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"t","version":"0"}}}' \
 '{"jsonrpc":"2.0","method":"notifications/initialized"}' \
 '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
 '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"crawl_website","arguments":{"url":"https://example.com"}}}' \
 | node index.js

With no APIFY_TOKEN set, request id 3 must return{"status":"error","error":{"code":"missing_token",...}} — proof the handler reaches thereal Apify boundary instead of inventing a result.

MCP Server · Populars

MCP Server · New

    frontman-ai

    frontman

    The AI agent that lives in your framework/browser

    Community frontman-ai
    Laksh-star

    Screenward

    Movie and TV discovery, watch planning, and streaming availability—for browsers and MCP clients.

    Community Laksh-star
    codescene-oss

    CodeHealth MCP Server

    The CodeScene MCP Server exposes CodeScene’s Code Health analysis as local AI-friendly tools.

    Community codescene-oss
    Fan-Pier-Labs

    OpenRecord

    Let Claude read and manage your Epic MyChart health records — labs, meds, messages, refills, imaging. Claude Desktop extension (MCP server), iOS app, and CLI. Runs on your device.

    Community Fan-Pier-Labs
    video-db

    skills

    Server-side video workflows for agents: ingest, understand, search, edit, stream.

    Community video-db