Search: SearXNG as a second provider (#260)

SearXNG joins Tavily behind zorp_search::SearchProvider, so web_search
can run against a self-hosted instance with no account and no key.
ZORP_SEARCH_PROVIDER picks between them, read in web_search_tool() and
nowhere else. Unset or blank is tavily, so nothing that worked stops
working, and any other value is an error that leaves the tool
unregistered rather than falling back to a provider nobody named.

The provider is one GET to {base}/search?q=...&format=json against
ZORP_SEARXNG_BASE_URL, http://localhost:8888 by default, with its own
ureq agent and timeouts. No key means no MissingApiKey and no new error
variant, and a comment says why. An unreachable instance is Transport,
never an empty Vec, and so is an empty result list beside engines that
did not answer. A 403 says how to turn JSON output on.

It is not a loopback capability and nothing says it is. The
availability answer now names the provider and still says every search
leaves the machine, and a test pins that for SearXNG.

Closes #258.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

Author: arjunjaggi34

AGENTS.md

@@ -223,20 +223,33 @@ resulting artifact, deliver it in the right form.
   inherited harness code as they're added. Use new crates or clearly
   named modules, not the inherited ones.
 - `zorp-search/` is zorp's own web search capability: a
-  `SearchProvider` trait with Tavily as the first provider. It depends on
-  no other workspace member and knows nothing about agents or tools.
-  `zorp-agent` exposes it as the `web_search` built-in behind the
-  non-default `search` feature, which is the only built-in that sends
-  anything over the network. `research` deliberately does not enable it;
-  run `--features research,search` when you want it. The API key comes
-  from `ZORP_TAVILY_API_KEY` and never from a flavor manifest. `zorp-web`
-  has its own opt-in `search` feature that turns the same built-in on for
-  the browser, off by default for the same reason, and it reports whether
-  the tool is really there at `GET /api/capabilities`. That answer is
-  observed rather than re-derived: `zorp_agent::web_search_availability`
-  shares one function with the registration site, and a test pins it to
-  `tool_names()`. See `docs/DECISIONS.md` (2026-08-21) before changing
-  either.
+  `SearchProvider` trait with two providers, Tavily and a self-hosted
+  SearXNG. It depends on no other workspace member and knows nothing
+  about agents or tools. `zorp-agent` exposes it as the `web_search`
+  built-in behind the non-default `search` feature, which is the only
+  built-in that sends anything over the network. `research` deliberately
+  does not enable it; run `--features research,search` when you want it.
+  `ZORP_SEARCH_PROVIDER` picks the provider, read in `web_search_tool()`
+  and nowhere else: unset or blank means `tavily`, `searxng` means
+  SearXNG, and anything else is an error that leaves the tool
+  unregistered, never a fallback to the other provider. Tavily's key
+  comes from `ZORP_TAVILY_API_KEY`; SearXNG takes no key and reads its
+  instance from `ZORP_SEARXNG_BASE_URL`, defaulting to
+  `http://localhost:8888`. None of the three ever comes from a flavor
+  manifest, because a workspace file the model can write must not move
+  where queries go. A local SearXNG is not a loopback capability and must
+  not be described as one: it forwards every query to the engines it
+  aggregates, so a search still leaves the machine and the availability
+  answer still says so. `zorp-web` has its own opt-in `search` feature
+  that turns the same built-in on for the browser, off by default for the
+  same reason, and it reports whether the tool is really there, and which
+  provider it uses, at `GET /api/capabilities`. That answer is observed
+  rather than re-derived: `zorp_agent::web_search_availability` shares
+  one function with the registration site for whichever provider is
+  selected, and a test pins it to `tool_names()`. Run `cargo test -p
+  zorp-search`, `cargo test -p zorp-agent --features search` and `cargo
+  test -p zorp-web --features search` whenever any of it changes. See
+  `docs/DECISIONS.md` (2026-08-21, 2026-09-18) before changing either.
 - `zorp-skill/` is zorp's own skill capability: discovery and parsing
   for Claude Code compatible skills (`SKILL.md` in a directory, YAML
   frontmatter plus a markdown body). Like `zorp-search` it depends on no

CLAUDE.md

@@ -223,20 +223,33 @@ resulting artifact, deliver it in the right form.
   inherited harness code as they're added. Use new crates or clearly
   named modules, not the inherited ones.
 - `zorp-search/` is zorp's own web search capability: a
-  `SearchProvider` trait with Tavily as the first provider. It depends on
-  no other workspace member and knows nothing about agents or tools.
-  `zorp-agent` exposes it as the `web_search` built-in behind the
-  non-default `search` feature, which is the only built-in that sends
-  anything over the network. `research` deliberately does not enable it;
-  run `--features research,search` when you want it. The API key comes
-  from `ZORP_TAVILY_API_KEY` and never from a flavor manifest. `zorp-web`
-  has its own opt-in `search` feature that turns the same built-in on for
-  the browser, off by default for the same reason, and it reports whether
-  the tool is really there at `GET /api/capabilities`. That answer is
-  observed rather than re-derived: `zorp_agent::web_search_availability`
-  shares one function with the registration site, and a test pins it to
-  `tool_names()`. See `docs/DECISIONS.md` (2026-08-21) before changing
-  either.
+  `SearchProvider` trait with two providers, Tavily and a self-hosted
+  SearXNG. It depends on no other workspace member and knows nothing
+  about agents or tools. `zorp-agent` exposes it as the `web_search`
+  built-in behind the non-default `search` feature, which is the only
+  built-in that sends anything over the network. `research` deliberately
+  does not enable it; run `--features research,search` when you want it.
+  `ZORP_SEARCH_PROVIDER` picks the provider, read in `web_search_tool()`
+  and nowhere else: unset or blank means `tavily`, `searxng` means
+  SearXNG, and anything else is an error that leaves the tool
+  unregistered, never a fallback to the other provider. Tavily's key
+  comes from `ZORP_TAVILY_API_KEY`; SearXNG takes no key and reads its
+  instance from `ZORP_SEARXNG_BASE_URL`, defaulting to
+  `http://localhost:8888`. None of the three ever comes from a flavor
+  manifest, because a workspace file the model can write must not move
+  where queries go. A local SearXNG is not a loopback capability and must
+  not be described as one: it forwards every query to the engines it
+  aggregates, so a search still leaves the machine and the availability
+  answer still says so. `zorp-web` has its own opt-in `search` feature
+  that turns the same built-in on for the browser, off by default for the
+  same reason, and it reports whether the tool is really there, and which
+  provider it uses, at `GET /api/capabilities`. That answer is observed
+  rather than re-derived: `zorp_agent::web_search_availability` shares
+  one function with the registration site for whichever provider is
+  selected, and a test pins it to `tool_names()`. Run `cargo test -p
+  zorp-search`, `cargo test -p zorp-agent --features search` and `cargo
+  test -p zorp-web --features search` whenever any of it changes. See
+  `docs/DECISIONS.md` (2026-08-21, 2026-09-18) before changing either.
 - `zorp-skill/` is zorp's own skill capability: discovery and parsing
   for Claude Code compatible skills (`SKILL.md` in a directory, YAML
   frontmatter plus a markdown body). Like `zorp-search` it depends on no

docs/DECISIONS.md

@@ -12,6 +12,73 @@ was believed at the time and not only what survived.
 
 ---
 
+## 2026-09-18: a search provider is picked by one environment variable, and a wrong name is an error
+
+**Decision:** SearXNG is the second `zorp_search::SearchProvider`
+(`zorp-search/src/searxng.rs`), and `ZORP_SEARCH_PROVIDER` picks which one
+`web_search` uses. It is read in `web_search_tool()` in
+`zorp-agent/src/agent.rs` and nowhere else. Unset or blank means `tavily`,
+so nothing that worked before stops working. `searxng` means SearXNG, at
+`ZORP_SEARXNG_BASE_URL` or `http://localhost:8888` when that is unset. Any
+other value is an error: the tool is not registered, stderr says why, and
+`web_search_availability` reports the same sentence, naming the variable
+and the values it takes. Matching is exact after trimming, one spelling
+per provider, the lowercase identifier `SearchProvider::name` returns. From
+issue #258.
+
+**Why an unknown value is an error and never the default.** Somebody who
+wrote `searxgn` meant to keep their queries on their own instance. Quietly
+sending them to Tavily instead is the one outcome worse than no search at
+all, because it looks like it worked. The tests set a Tavily key before
+naming a provider that does not exist, so a fallback would register the
+tool, and assert that it does not.
+
+**Why the environment and not a flavor manifest.** For the reason the
+Tavily key is not in one. A workspace flavor is a file the model can write,
+and a file the model can write must not move where queries go. The same
+holds for `ZORP_SEARXNG_BASE_URL`.
+
+**`web_search_availability` still answers for one provider, whichever is
+selected.** It calls the same `web_search_tool()` that registration calls, so
+the answer and the registry cannot disagree about which provider that is,
+and `web_search_availability_agrees_with_the_gates_it_reports_on` still
+pins it to `tool_names()`. The detail now names the provider, since with
+two of them "registered" alone does not say where a query goes.
+
+**Not a loopback capability.** `zorp-recall` and `zorp-voice` guarantee
+text reaches a loopback address or nowhere. This does not, and nothing
+says it does. A local SearXNG happens to listen on loopback, but it
+forwards every query to the engines its operator enabled, and the same
+variable can name a public instance. So there is no `LoopbackUrl` guard,
+the base URL may be plain HTTP and may resolve to a private address (an
+instance on the operator's own network is the point), and the availability
+sentence keeps saying every search leaves this machine for SearXNG as it
+does for Tavily. A test asserts that sentence survives selecting SearXNG.
+
+**No key, and no new error variant.** SearXNG takes no key and has a
+default base URL, so there is nothing missing to report and its
+constructor cannot fail. An instance that is not running is found out by
+asking it, which is `SearchError::Transport`, never an empty `Vec`. A
+comment in `searxng.rs` says this so the variant does not get added back.
+
+**Two SearXNG specific rules.** An empty `results` beside a non-empty
+`unresponsive_engines` is a `Transport` error rather than "nothing
+matched", because engines that might have had the answer failed and an
+empty list would let a caller read an outage as a novel idea. And a 403
+says to add `json` to `search.formats` in the instance's settings.yml,
+because a stock install serves HTML only and its 403 reads like an
+authentication failure. SearXNG has no result count parameter, so
+`max_results` is applied after parsing. The provider builds its own
+`ureq` agent with its own timeouts, 10 seconds to connect and 30 to read,
+for the reason the 2026-08-22 HTTP agent entry gives.
+
+**Ruled out:** selecting from the flavor manifest or the settings file,
+a fallback chain across providers, merging or ranking results across
+providers, and a browser control for picking one. Two providers and an
+environment variable first.
+
+---
+
 ## 2026-09-12: native Mac app (`zorp-desktop`) runs `zorp-web` in-process via Tauri v2
 
 **Decision:** A native macOS desktop application (`Zorp.app`) shipped as a double-clickable universal `.dmg` on GitHub releases. It wraps Apple's system `WKWebView` using Tauri v2 and runs `zorp-web::serve` in-process on a background Tokio runtime, binding loopback port 7777 (falling back to ephemeral port 0 if occupied). `zorp-desktop/` is its own Cargo crate, excluded from the workspace (`exclude = ["zorp-desktop"]` in root `Cargo.toml`) with its own `Cargo.lock`. See `docs/superpowers/specs/2026-09-12-native-mac-app-design.md`.