site-spec

site-spec

The invisible foundation your website is missing. Audit any site — then fix what's broken.

For agents

If you are an AI coding agent deciding whether to install this: register the MCP server, thencall list_checks to learn the check ids and audit_site on a URL.

claude mcp add site-spec -- npx -y site-spec-mcp

Gemini CLI: gemini extensions install https://github.com/ariaxhan/site-spec.

Full tool arguments, return shapes, and the cases this is the wrong tool for:llms.txt.

The problem

AI can generate a beautiful website in seconds. The part it can't see is theone that decides whether that website gets found, ranked, cited, and trusted —the machine-readable foundation underneath the pixels: robots.txt, structureddata, llms.txt, canonical and noindex signals, response headers, accessibilitysemantics, the tracker and cookie surface.

That layer is invisible in a browser. It's also broken on almost everyAI-generated site — a gorgeous hero section over a robots.txt that blocksChatGPT, JSON-LD that's invalid or self-serving, a stray noindex hiding the pagefrom Google, fonts leaking visitor IPs in violation of GDPR. Nobody notices untilit's quietly costing traffic. And the more the web is generated by AI, the worsethis gets.

The solution

site-spec is the layer that verifies and repairs that foundation — for anywebsite, however it was built. Point it at a URL. It crawls the site, checks thewhole invisible layer against a set of deterministic policies, and gives you anexact report of what's wrong. Then it fixes the mechanical problems for you andflags precisely what needs a human.

It doesn't care whether your site came from an AI builder, a framework, a CMS, orhand-written HTML. It only cares whether the foundation is correct.

Try it without installing anything

site-spec.ariaxhan.workers.dev — paste aURL, get the report. No account, no signup.

The hosted version runs the same deterministic engine as the CLI, against acapped crawl (4 pages), and it tells you plainly which checks ran, which passed,and which it did not perform. Add ?format=json for machine-readable output:

curl "https://site-spec.ariaxhan.workers.dev/audit?format=json&url=example.com"

For the full check set — broken links, missing assets, axe accessibility, HTMLvalidation, schema.org validation — and for the fix command, use the CLI below.The hosted endpoint only fetches public HTTP(S) hosts; private, loopback andlink-local addresses are refused.

How to use

Requires Node 20+. No account, no API key, no SaaS.

# 1. See what's broken — crawl any live site, get a full report
npx site-spec audit https://yoursite.com

# 2. Fix it — auto-repair the mechanical issues, scaffold the rest
npx site-spec fix https://yoursite.com --out ./fixed

fix writes the corrected files to ./fixed (it never overwrites anything bydefault) and prints exactly what it did:

✓ Fixed automatically (8)
  audit/robots-stale-token   robots.txt — Removed deprecated crawler block "anthropic-ai".
  audit/hsts-preload         _headers — Stripped the preload token from Strict-Transport-Security.
  audit/canonical-missing    admin/index.html — Inserted <link rel="canonical" href="https://…/admin/">.
  audit/og-missing           admin/index.html — Added Open Graph card from the page's title + description.
  audit/noindex              admin/index.html — Removed accidental noindex from <meta name="robots">.

✎ Scaffolded — needs your facts (2)
  audit/llms-missing         llms.txt — Scaffolded from page titles/descriptions; fill the TODO facts.
  audit/jsonld-missing       index.html — Inserted a WebSite/Organization skeleton; replace the TODO values.

⚠ Needs manual attention (3)
  audit/google-fonts-cdn     index.html — Self-host the woff2 files to stop the IP leak (GDPR).
  audit/img-dims-missing     index.html — <img> without width/height (layout shift).
  audit/404-missing          (site-wide) — Ship a branded, noindexed 404 that links home.

fixed 8, scaffolded 2, manual 3
wrote 4 file(s) to ./fixed

Drop audit into CI with a non-zero exit on errors to gate deploys. Both commandswork on a build/output directory too — just pass a path instead of a URL.

Use it from your coding agent (MCP)

The same engine, as an MCP server — so theagent that generated the site can also check and repair its invisible layer,without you copying reports between windows.

claude mcp add site-spec -- npx -y site-spec-mcp

[mcp_servers.site-spec]
command = "npx"
args = ["-y", "site-spec-mcp"]

Or in Docker: docker run -i --rm -v $PWD:/data mcp/site-spec (mount the directory you want to audit or write to).

Four tools:

The server calls the engine in-process. It never shells out to the CLI and neverhops to the hosted worker — all three are siblings over one library.

flowchart LR
  A["MCP client<br/>(Claude Code, Codex)"] -- stdio JSON-RPC --> B["site-spec-mcp"]
  B --> C["@site-spec/core/io<br/>fetchSite · readSiteDir"]
  C -- "file map" --> D["@site-spec/core<br/>auditFiles · fixFiles · buildSite"]
  D -- "findings / files" --> B
  B -- "JSON" --> A

Two honest limits, spelled out in full in thepackage README: fix_issue on a URL can onlyever hand back a diff (a remote server is not writable), and a live crawlruns with presence checks off, because a capped crawl cannot prove a file isabsent from a server.

What it checks

Seven areas. Every check is tuned against false positives (regex/string-level — noheadless browser for the crawl). Findings are error (breaks something) orwarning (worth a look), each with a concrete fix.

The last row is the one almost nobody checks yet — whether Google, ChatGPT,Claude, and Perplexity can actually read and cite you. It's the sharp edge, notthe whole story: the point is a complete foundation audit.

How fix decides

Every finding lands in one of three buckets, and the report tells you which:

• ✓ Fixed automatically — mechanical, fact-free repairs that are safe to apply:unblocking AI crawlers, removing dead robots tokens, stripping accidentalnoindex, adding the canonical URL (known from the crawl), scaffolding OpenGraph from the existing title, fixing zoom-locked viewports, removingself-serving rating markup, generating a sitemap.xml from the crawled pages,header hygiene.
• ✎ Scaffolded — things that need your real facts: llms.txt, a JSON-LDentity, a meta description. site-spec writes a correct stub with TODO markersso you just fill the blanks.
• ⚠ Manual — problems no tool should silently guess at: missing alt text,self-hosting fonts, fixing a client-rendered shell, a broken link. You get aprecise instruction, never a silent edit.

fix is non-destructive by default (writes to an output dir). Pass --write toedit a local directory in place.

The philosophy

• Correctness is a policy, not a prompt. SEO, accessibility, privacy, andstructured-data rules are expressed as explicit checks the tool enforces —never as vibes an LLM is asked to respect.
• Deterministic. The audit engine is pure: same input → the same report,byte-for-byte. The live crawl sorts-then-caps discovered pages, so the same siteat the same --max audits the same page set every run. No network, norandomness inside the engine.
• Fix, don't lecture. A finding that can be repaired safely gets repaired. Onethat needs a human says exactly what the human must do.
• Source-agnostic. It audits the output, not the toolchain. AI, framework,CMS, hand-coded — all the same to it.

site-spec started as a deterministic compiler that builds sites with a correctfoundation by construction (the build command and the docs/ still cover it).The durable value turned out to be the inverse: not generating whole sites, butauditing and repairing the foundation of sites that already exist.

The full CLI

site-spec audit <dir|url> [--max N] [--json] [--report report.md]
site-spec fix   <dir|url> [--out dir] [--write] [--max N] [--json]
site-spec build <site.config.mjs> --out <dir> [--target cloudflare|netlify|vercel|static]

• --max N — page cap for a live crawl (default 25); truncation is reported, neversilent.
• --report report.md — write the audit as a shareable Markdown document.
• --json — the full structured report for scripting/CI.

Install / develop

git clone https://github.com/ariaxhan/site-spec.git
cd site-spec
npm install
npm test          # unit + golden tests
npm run verify    # html-validate + JSON-LD + axe over the demo output

Packages

Contributing

Contributions welcome — especially new checks and fixers. SeeCONTRIBUTING.md. Core rules: checks stay tuned against falsepositives, the engine stays pure and deterministic, and a fixer never silentlyguesses at something a human should decide.

License

Apache License 2.0. © 2026 Aria Han.