Verified Regex Generator

Tests FastAPI

MCP → Verified Regex Generator. The user describes a pattern in plain English. An LLMgenerates the regex. An agent generates real sample strings (positive + negative examples),tests the regex against them with the actual re engine, and iterates if it's wrong.Regex is infamous for looking right while being subtly wrong — verification is the wholevalue here, not the generation.

Runs on Groq (openai/gpt-oss-120b by default) — fast and free-tierfriendly, so it's cheap to demo publicly.

This isn't "ask an LLM for a regex and hope." It's a closed verification loop:

description  ──►  LLM writes a candidate regex
                        │
description  ──►  LLM writes real test strings (incl. tricky near-misses)
                        │
                        ▼
              Python's `re` engine checks the candidate
              against every test string — ground truth,
              not the model's opinion of itself
                        │
              ┌─────────┴─────────┐
          all pass             something failed
              │                     │
              ▼                     ▼
           done               feed the exact failures
                               back to the model, try again

Demo

A real run: "US phone number" → the agent writes 16 test strings, proposes a candidate regex,checks it against Python's real re engine, and converges on a verified pattern in 2 iterations.

Demo

Full page + live tester screenshot

Screenshot

What's in here

regex_agent/core.py — the actual agent loop (model-agnostic of transport). Everythingelse is a thin wrapper around this.
mcp_server/ — a real MCP server exposinggenerate_verified_regex as a tool, so it can be used directly from Claude Desktop orClaude Code.
web/ — a FastAPI + vanilla-JS demo with two parts:
1. A live, animated view of the agent's reasoning: test cases, each candidate regex, andthe pass/fail table per iteration.
2. A regex101-style live tester underneath — an editable pattern field with flags(g/i/m/s/full-match), live match highlighting against your own test string, amatch list with capture groups, and a plain-English token-by-token breakdown of theregex (all client-side, no API calls). It auto-fills with whatever the agent justverified, but works standalone for any regex you paste in — useful even if you alreadyknow regex and just want to test one.
No MCP client required — runs in a browser.

Setup

pip install -r requirements.txt
cp .env.example .env   # then add your GROQ_API_KEY

Run the web demo

cd web
python server.py

Open http://127.0.0.1:8000.

Run the MCP server

Add this to your MCP client config (e.g. Claude Desktop's claude_desktop_config.json,or .claude/settings.json for Claude Code):

{
  "mcpServers": {
    "verified-regex-generator": {
      "command": "python",
      "args": ["/absolute/path/to/2 project/mcp_server/server.py"]
    }
  }
}

Then ask Claude something like "Use the verified regex generator to build me a regex for aUS phone number." — it will call the tool, which runs the full generate → test → verify loopserver-side and returns a JSON report.

Why this is a good showcase

Most "AI writes code" demos stop at generation. This one treats the LLM's first answer as ahypothesis, not an answer — and only claims success once it's checked against ground truth(the real regex engine, on real strings, including adversarial near-misses the model itselfproposes). That loop — generate → verify → revise — is the core pattern behind reliableagentic tools, and it's small enough to read end-to-end in regex_agent/core.py.

Verified Regex Generator