Repository avatar
Search & Data Extraction
v0.13.0
active

webrouter

io.github.Nimbleway/webrouter

Search, extract, answer, research and browse the web through one interface, across many providers

Documentation

WebRouter

WebRouter

The open source standard for agent web access.
One typed interface for search, extract, answer, research, live data and browsers, across two dozen providers.

PyPI License: Apache 2.0 Python 3.11 to 3.13 CI Join WebRouter on Discord

Documentation · Quickstart · Providers · Examples · Discord

Developer preview: the API may change before 1.0. Share feedback on Discord.

Every web API for agents has its own SDK, parameter names and response shape. WebRouter puts one interface in front of all of them, so switching provider is one argument, a failing provider falls back to the next, and your agent code never changes.

from web_router import WebRouter

with WebRouter() as router:  # no API key needed to start
    results = router.search("what is retrieval augmented generation", max_results=5, timeout=10)
    pages = router.extract(results.urls[:3], timeout=30)  # markdown, whichever backend serves it

print(results.provider, [doc.url for doc in pages.documents])

Why WebRouter

  • One call shape, every provider. The same request and the same typed response from Tavily, Exa, Brave, Perplexity, Firecrawl, Nimble and the rest. Nothing is lost: unmapped fields stay in extra and the raw payload in raw.
  • Never silently weaker. An option a provider cannot honour raises UnsupportedParameterError instead of being dropped, so a call never runs under looser constraints than you asked for.
  • Routing based on published benchmarks. auto tries the providers you hold keys for in the order committed benchmark results rank them, and falls back on an error or an empty answer. response.provider always says who answered.
  • Cost control. Every response says what it cost and how long it took. Cap a call or a whole session in dollars and seconds, try the free backends first, and keep a log of what you spent (budgets).
  • Keyless to start. Search, extract and a local browser work with no API key. Every key you add unlocks more providers and fallbacks.
  • Built for agents. An MCP server, plus ready-made tools for LangChain, Claude and OpenAI agents.
  • Measure, don't guess. benchmark() and evaluate() compare providers on your own queries and score their accuracy against known answers.
  • Yours to run. Apache 2.0, runs in your process with your own keys, and sends each key only to its own provider.

Install

pip install "webrouter[all]"     # or: uv add "webrouter[all]"

Python 3.11 or later. Extras per method (search, extract, research, answer, browser, mcp, ...) keep the install small; see the Quickstart.

Keys are read from the environment. Copy .env.example, which names every variable and where to get it, and fill in the ones you hold.

Methods

MethodWhat it doesKeyless
searchWeb results for a queryYes
extractPage contents for URLs, in markdown, html or textYes
answerA written answer with its sources, in one callNo
researchA multi-step research run, as prose or a schema you giveNo
live_dataA vendor's ready-made scraper for one site, returning recordsNo
open_browserA browser session you drive, local or in the cloudYes (local)
browseA task handed to a hosted browser agentNo
call_vendorA vendor's own API (Google Ads, Datadog, ...) with your credentialsNo
with WebRouter() as router:
    router.answer("who won the 2022 world cup final")  # .answer, .citations
    router.research("how did nvidia's datacenter mix shift in 2025", output_schema=MySchema)
    router.live_data("amazon_products", {"url": product_url}, provider="brightdata")

    with router.open_browser() as browser:  # a local Chromium, no key
        browser.goto("https://example.com")

Every method has an async twin on AsyncWebRouter.

Providers

MethodProviders
SearchNimble, Tavily, Exa, Brave, Perplexity, Parallel, Linkup, You.com, WebIQ, Keenable, DeepSeek, AWS AgentCore, Valyu, Seltz, Octen, plus the keyless You.com, Exa and Parallel MCP endpoints
ExtractNimble, Tavily, Firecrawl, Exa, Linkup, Oxylabs, Bright Data, Jina Reader, WebIQ, Keenable, DeepSeek, Browserbase
AnswerPerplexity, You.com, Linkup, Parallel, Exa, Brave
ResearchNimble, Parallel, Perplexity, Linkup, You.com
Live dataNimble, Apify, Bright Data, Oxylabs
Browser sessionsLocal Chromium, Browserbase, Steel, Hyperbrowser, Anchor, Kernel
Browser agentsBrowserbase, Browser Use, Skyvern

Providers has each provider's key, the options it maps and how it was verified. Adding one is two methods: see Adding a provider.

Routing and fallback

For search, extract, answer, research and the browser methods, name no provider and auto serves the call: every provider you hold a key for, in the order committed benchmark results rank them, then the free tiers. A member that fails, is rate-limited or answers empty passes the call to the next. A member that cannot honour an option you set is skipped for that call, and if none can, the call raises rather than running without it. Live data has no fallback, since a source belongs to one vendor: live_data_sources() with no provider searches every vendor you hold a key for, and each source it returns names its own, which live_data() runs it on. Vendor calls always name their vendor, since an account belongs to one.

from web_router import Provider, WebRouter

WebRouter(Provider.TAVILY).search("...")  # one provider, by name
WebRouter().search("...", provider="exa")  # or per call

router = WebRouter(provider_config={"auto": {"chain": {"extract": ["nimble", "firecrawl"]}}})
router.extract(url)  # Nimble, then Firecrawl if Nimble fails

A named provider is never quietly swapped for another: a missing key raises an error naming the variable to set. In a service, share one router per process, set timeout= on each call and turn on the web_router logger: see Running in production. The auto order lists each method's order and how it is generated.

Budgets and cost control

from web_router import Budget, WebRouter, prices

router = WebRouter(router="cheapest_first")  # free backends first, then the cheapest paid ones
log = router.log_spend("spend.jsonl")  # from here on, every answered call is logged
response = router.answer("...", budget=Budget(max_cost_usd=0.01, time_limit_s=10))
response.provider, response.cost_usd, response.cost_source, response.elapsed_s

with router.budget(Budget(max_cost_usd=3.0, time_limit_s=600)) as b:
    b.search("...")
    b.research("...")
    b.status()  # spent, reserved, remaining, time left

log.report()  # what you spent, by day, backend and method
prices.estimate("tavily", "search")  # 0.008: list price of one request

Every response carries cost_usd: the backend's own figure where it states one (cost_source="reported"), else an estimate. Unless a router ranks the backends itself (the default rules router does not), a cost cap makes auto try the cheaper of the backends that score alike first.

Each attempt is checked against what is left before it starts, so parallel calls never overspend together, and a backend that does not fit is skipped with the reason recorded. Estimates come from the list prices shipped in web_router.prices, not your bill. Only search (its keyless backends) and open_browser (local) have a $0 backend; Cost and budgets has the table per method, a fall-back for a batch that runs out, and the spend log. An experimental router can also pick the method and provider for each prompt, and url_policy="public_only" keeps a planned call off private and cloud-metadata hosts. See Smart routing.

Use it from an agent

MCP server. Every method as a tool, running on your machine with your keys:

claude mcp add web-router -- uvx --from "webrouter[mcp]" web-router-mcp

The MCP guide covers Claude Desktop, Cursor and other clients, credentials, tool costs and browser guards.

Agent frameworks. Ready-made web_search and web_extract tools, with a compact rendering for the model and the full response kept for you:

from web_router import WebRouter
from web_router.integrations.langchain import web_router_tools

tools = web_router_tools(WebRouter(), max_results=5)  # LangChain

web_router.integrations.anthropic and web_router.integrations.openai give Claude and OpenAI agents the same tools as plain tool definitions. See Agent integrations.

Compare providers on your own queries

report = router.benchmark("nvidia q3 earnings", providers=["nimble", "tavily", "you-mcp"])
for run in report.runs:
    print(run.provider, run.latency, run.count if run.ok else run.error)

benchmark_extract() does the same for extract, and evaluate() scores each provider's accuracy on a question set with known answers (SimpleQA, FRAMES, SealQA or your own CSV) with an LLM judge. See Benchmarks and evaluation, and benchmarks/ for the published runs behind the default order.

WebRouter or calling each provider directly

Each provider's SDKWebRouter
Switching providerRewrite the call, options and parsingOne argument
Response shapeDifferent per providerOne typed model, raw payload kept
An unsupported optionVaries by providerRaises, naming who can serve it
A provider failsYour retry codeFalls back to the next you hold a key for
Which provider to useGuessworkOrder from published benchmarks, benchmark() on your queries
SpendPer-vendor dashboardsPer-call and per-session caps, list prices in the package
Agent toolsOne integration per providerOne MCP server and one set of tools

Documentation

Get startedMethodsGo further
IntroductionSearchAgent integrations
QuickstartExtractSmart routing
How it worksAnswerBenchmarks
ProvidersResearchAdding a provider
ConfigurationLive dataMCP server
ErrorsBrowserExamples
Vendor dataChangelog

Contributing

Contributions are welcome, and the most useful one is small: map one more of a provider's options, or add a provider.

make install  # every extra, dev tools and pre-commit hooks
make check    # lint, typecheck and the offline unit tests; no keys needed

CONTRIBUTING.md has the workflow and the provider checklist, STYLE_GUIDE.md the review standards, and AGENTS.md the rules for AI coding agents working on this repo.

Community

Useful? Star the repo so more agent developers can find it. ⭐

License

Apache License 2.0. The SimpleQA grader prompt vendored at src/web_router/evals/prompts/ keeps its own MIT license, in LICENSE-simple-evals beside it.

PYPI
webrouter
Install Command
pip install webrouter
Runtime: uvx