webfetch

webfetch

Web search for LLM agents that you run yourself - up to 8x fewer inputtokens and 3x lower cost than hosted web_search, at the same accuracy.

Hosted web-search tools charge $10 per thousand searches and then bill youagain for every token of retrieved content they push into your contextwindow. webfetch replaces them with a local pipeline - multi-enginesearch, page fetching and extraction, semantic reranking, sentence-levelcompression - exposed as a web_search tool your model calls like anyother. And unlike every hosted tool and search API we surveyed, repeatedand paraphrased queries are served from a semantic cache for free.

(Install with pip install webfetch-llm; the import name is webfetch.)

Jump to:The headline ·What you get ·Getting started ·Check your setup ·Full benchmark results ·Claude Code ·Agent loop ·Savings report ·How it works ·Caveats

Same accuracy. A third of the cost. An eighth of the tokens.

One agent loop, one model, one judge, 50 SimpleQA questions. The onlything that changes between rows is the search tool:

Swap Opus for gpt-5.6-sol and the same webfetch tool hits 96% - hostedparity - at $0.040/query and 2,156 tokens: an eighth of what the hostedtool pushes into your context. Full resultscover every arm we ran.

These numbers are the WORST case for webfetch - measured on an emptycache. In real use the gap widens on its own: repeats and rewords servefrom cache for free, and the token advantage is paid again on everylater turn that keeps search results in context. It adds up to receiptslike this one, from an ordinary Claude Code session:

Every claim in this README is generated by an eval harness that ships inthis repo - the question sets, per-question records, judging protocol,and the negative results are all in evals/, and every table canbe regenerated with one command. Don't take our word for the grading:evals/results/README.md maps every table rowto the raw result file that produced it, down to per-question judgeverdicts.

What you get

A search pipeline you own (4-engine RRF fusion, local extraction,sentence-level compression). Results come from reciprocal-rank fusionacross DuckDuckGo, Brave, Serper, and Tavily - whichever of them you havekeys for. DDG needs no key, so the tool works at literally zero cost outof the box; every key you add joins the fusion automatically. Pages arefetched and extracted locally (trafilatura, readability, newspaper4k,Playwright rendering for JS pages and 403 walls), chunked, ranked by ahybrid BM25 + bi-encoder cascade with a cross-encoder on top, thencompressed to the sentences that answer the query - measured 50% fewertokens at zero recall loss.

Caching nobody else has (exact + semantic matching, volatility-awareTTLs). Two layers in one sqlite file: page text byURL, ranked results by query. Identical queries hit an exact cache.Paraphrased queries hit a semantic cache - an embedding shortlistverified by an NLI cross-encoder, tuned eval-first for precision (zerowrong-target matches across every live run we have done). Cache lifetimesadapt to the query: prices and scores expire in 15 minutes, current-ishtopics in 7 days, release notes and specs in 90 - classified by thecalling model's hint or a local classifier. The model sees provenance onevery cached result ([cache: semantic match to "...", 2h old, recent])and can send force_fresh when it disagrees. No hosted tool or searchAPI we surveyed offers any client-visible caching at all.

The model can maintain its own cache (save_finding, labeledUNVERIFIED, kill switch included). If a search comes up empty andthe model answers from some other source (a hosted search fallback, say),it can call save_finding to store what it learned - markedmodel-contributed, served with an explicit UNVERIFIED warning and aforce_fresh escape hatch, and aged out on the normal TTL rules. A killswitch (SAVE_FINDING_ENABLED = False) exists for deployments that neverwant unverified content cached.

Levers where models actually need them (full_results, fetch_url,freshness, force_fresh). full_results returnsuncompressed excerpts for list and ranking queries (compression trimsparallel list items - we measured it). fetch_url pulls one cited pagein full, instantly if the pipeline has ever fetched it. freshness hintscontrol cache lifetime; force_fresh bypasses it.

Receipts (session and lifetime, exact counters). Usage counterspersist in the cache file;webfetch-savings (or the savings_report tool) shows what you did notpay hosted-search pricing for, split by this session and lifetime.Sample below.

Getting started

The one-liner (uv installs the package on first launch, @latest picksup new releases automatically):

claude mcp add webfetch -- uvx --from webfetch-llm@latest webfetch-mcp

Or manage the install yourself, two commands:

pip install "webfetch-llm[all]"
claude mcp add webfetch -- webfetch-mcp

The slim install (webfetch-llm without extras) starts in seconds andruns degraded: BM25 ranking, exact-only cache, lexical compression.[all] pulls the semantic stack (torch - a few minutes once) plusPlaywright, PDF, and table extraction; it is the configuration everybenchmark number in this README was measured on. Needs pip >= 24 in freshvenvs (python -m pip install -U pip - older pips crash on a duplicatedextra in our dependency tree). If you install into a venv, register theabsolute path to webfetch-mcp or keep the venv active.

Keys are optional (DDG works with none) but improve recall - all theengines have free tiers:

cp .env.example .env    # fill in what you have

The console scripts and the MCP server pick up a .env from thedirectory they run in; exported env vars and --env flags onclaude mcp add always take precedence. import webfetch as a librarynever reads .env - your process env is yours.

Check your setup

webfetch-status (or asking the model to call the status tool) answers"is my key being picked up, and what am I actually running?":

webfetch 0.1.3

search engines:
  ddg      ready (no key needed)
  brave    ready (BRAVE_API_KEY is set)
  serper   off   (SERPER_API_KEY not set)
  tavily   ready (TAVILY_API_KEY is set)

active provider: multi(ddg+brave+tavily) (RRF fusion)

optional features:
  semantic ranking/cache/compression: on
  JS-page rendering (playwright):     on
  PDF extraction (pdfplumber):        OFF (pip install 'webfetch-llm[pdf]')
  HTML tables (pandas+tabulate):      on
  result compression:                 on (crossencoder)

cache: /Users/you/.webfetch/cache.db
  5.4 MB, 70 lifetime searches (webfetch-savings for the receipt)

Key names only - values are never printed. Configuration is environmentvariables, read at server start:

With zero keys the default resolves to plain DDG; the status output saysso honestly rather than calling it fusion. Library users can skip enventirely: Pipeline(search=get_search_adapter("fallback"), cache=...).

Full benchmark results

Every arm, same 50 SimpleQA questions, same judge, one same-day run(2026-07-14). Agent-loop arms share an identical loop; only the searchtool differs. Costs include model tokens plus each provider's publishedper-search fees; ours include estimated engine fees.

* Haiku's failures were mostly re-searching past the turn cap, and itslast few questions ran on a degraded engine set after we exhausted a freetier mid-benchmark. Treat it as a floor.

Notice the token column: webfetch results cost half the input tokens ofthe snippet APIs and a fifth to an eighth of the hosted tools - which iswhy our cost stays lowest even where per-search fees are similar.

On a second dataset of 27 questions about events from the two weeksbefore the run (hand-written, never published, so no vendor could havetuned on them), webfetch scored 100% with fusion and 100% with DDG alone;the hosted tools also scored 100%. Fresh events are not the hard part -the date-injection trap is.

Reproduce: python evals/run_e2e_eval.py --arms ours-multi,hosted - theharness, datasets, and per-question records are in evals/. Thetable above comes frome2e_eval_20260714_005531.json(plus the Haiku run and the fresh-events set - the full claim-to-file mapis in evals/results/README.md). The questionsare a seeded sample (seed 42) of OpenAI's public SimpleQA set, sobuild_datasets.py regenerates the identical 50 on your machine. Onehonest caveat: the live web moves between runs, and at n=50 the 96-vs-92accuracy gap is 48-vs-46 correct - parity within noise. The token andcost columns are the mechanically stable result.

Using it in Claude Code / Claude Desktop

Fastest route in Claude Code - install it as a plugin, two slashcommands (runs the same uv one-liner under the hood):

/plugin marketplace add firish/webfetch
/plugin install webfetch@webfetch

Or register the MCP server yourself. Either way, anew session shows webfetch under /mcp with five tools: web_search, fetch_url,save_finding, status, and savings_report. Ask anything recent andwatch the tool calls; ask the same thing reworded and the result comesback with a [cache: semantic match ...] header instead of a freshsearch. First call in a session is slow (encoder warm-up plus real pagefetches, 10-40s); cached calls are instant.

Toggle the server off per-session in /mcp; remove it with/plugin uninstall webfetch or claude mcp remove webfetch, matchinghow you installed it. Your cache and its receipt history live in~/.webfetch/ and survive reinstalls. Run one server per machine - thesemantic cache assumes a single process owns its file.

Update notice: the server makes one request to pypi.org per process tocheck for a newer release, surfaced as one line in savings_reportoutput. UPDATE_CHECK_ENABLED = False in webfetch/config.py disablesit; nothing else is ever sent anywhere.

Use it in your own agent loop

import time
import anthropic
from webfetch import WEB_SEARCH_TOOL, handle_web_search

client = anthropic.Anthropic()
messages = [{"role": "user", "content": "What did the FOMC decide this week?"}]
system = (f"Today's date is {time.strftime('%Y-%m-%d')}. "
          "Use web_search for recent facts.")

while True:
    response = client.messages.create(
        model="claude-opus-4-7", max_tokens=2000, system=system,
        tools=[WEB_SEARCH_TOOL], messages=messages,
    )
    if response.stop_reason != "tool_use":
        break
    messages.append({"role": "assistant", "content": response.content})
    results = [{"type": "tool_result", "tool_use_id": b.id,
                "content": handle_web_search(b.input)}
               for b in response.content if b.type == "tool_use"]
    messages.append({"role": "user", "content": results})

A complete version with prompt caching and adaptive thinking is inexamples/agent_loop.py. The tool schema isprovider-agnostic in spirit - OpenAI function calling needs only amechanical reshape of the dict.

Put today's date in your system prompt. This is not optional. Modelsrefuse to search for events they believe have not happened yet: on ourfresh-events dataset, loops without the date declined to even call thetool on up to 10 of 27 questions ("Wimbledon 2026 hasn't taken placeyet"). One line fixes it. Hosted search tools do this server-side, whichis part of why nobody notices until they run their own tool.

handle_web_search never raises. Engine failures, empty results, andmalformed input come back as readable strings the model can react to,because an exception mid-conversation kills the whole agent loop.

The savings report

Counters accumulate in the cache file as you search. Ask the model to runthe report, or run webfetch-savings yourself. The MCP tool splits thissession from lifetime, so you can see what one conversation saved:

That one-liner unpacked - a 30-search session on this machine:

Counters are exact; the dollar lines are estimates with the assumptions(hosted per-search fee, hosted tokens per call - both measured - and yourmodel's token price) exposed as arguments to webfetch.savings_report().

How it works

query -> multi-engine search (RRF fusion, circuit breakers)
      -> semantic query cache? serve with provenance header
      -> fetch pages concurrently (page cache by URL)
      -> extract (trafilatura -> readability -> newspaper4k -> playwright)
      -> chunk (400 chars) -> BM25 + bi-encoder fusion -> cross-encoder top 5
      -> compress to query-relevant sentences -> source-labeled result

Everything heavy is optional: without [rerank] you get BM25 ranking,exact-match caching, and lexical compression, with a logged warninginstead of an ImportError. DDG deserves a special note: itfingerprint-blocks automated clients with silent empty responses.webfetch detects that (empty-with-peers in fusion, any empty in thefallback chain), benches the engine on a circuit breaker, and routesaround it; the TLS side is handled by the ddgs dependency.

Design decisions and their measurements are documented as they happenedin docs/architecture.md anddocs/ROADMAP.md - including the negative results.

Caveats, honestly

• Latency. A fresh search takes 10-40 seconds - real pages get fetchedand ranked locally. Hosted search returns in ~10s, snippet APIs in ~6s.Cache hits are instant. If you need sub-second search and do not careabout cost, this is not your tool.
• The cache is single-process. One agent loop, one MCP server, or onenotebook at a time; a shared team cache backend is on the roadmap.
• Engine free tiers are real quotas. We exhausted Brave's monthly tierduring benchmarking. The resilience layer degrades gracefully, but yourrecall degrades with it. Engine fees in the tables are estimates frompublished prices.
• Answer quality depends on the model driving the tool - weak modelsformulate worse queries and re-search instead of reading (see the Haikurow).
• Semantic-cache recall on cross-form rewords ("what did X add" vs"X new features") is partial: measured 6/10 on real pairs after tuning,because some paraphrase forms defeat every available verifier. Missescost a re-search, never a wrong answer.

Benchmarks and data

Three eval layers, all in evals/: an offline matcher eval thatpicks the semantic-cache thresholds, a live retrieval eval (recall,tokens, cache diagnostics), and the end-to-end answer eval quoted above(SimpleQA protocol: normalized exact match, then an LLM judge). Datasetsare built deterministically from SimpleQA (MIT), QQP/GLUE, and FreshQA(CC-BY-SA), with provenance sidecars next to each file. The fresh-eventsset was hand-written against verified news sources days before thebenchmark ran, specifically so no model or vendor pipeline could haveseen it.

License

MIT.