yoetz ask should support xAI Agent Tools API (x_search) for live X / Twitter search via Grok

[avivsinai]

Problem

yoetz ask --provider openrouter --model x-ai/grok-4 (and x-ai/grok-4.20) cannot perform live X search even though the model has web_search: true in the registry. Grok responds with "I cannot invoke a search tool in this call" and Live X data: no.

Root cause: yoetz routes Grok through the OpenRouter Chat Completions endpoint, but xAI's older Live Search API was deprecated (error message) and the new path is the Agent Tools API on https://api.x.ai/v1/responses with {"tools":[{"type":"x_search"}]}. The Chat Completions / OpenRouter route does not expose those tools.

Repro

yoetz ask --provider openrouter --model x-ai/grok-4.20 --format json \
  -p 'Use live X search to find Hebrew posts about רישיון רחפן. Return URLs.'

Response: "I cannot invoke a search tool in this call. Live X data: no. Number of distinct posts cited: 0".

What actually works (workaround used today)

Direct curl to xAI's Responses API:

curl -sS https://api.x.ai/v1/responses \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "grok-4.3",
    "input": [{"role":"user","content":"Search X for Hebrew posts about רישיון רחפן. Return URLs."}],
    "tools": [{"type": "x_search"}],
    "max_output_tokens": 4000
  }'

Result: 6 x_search tool calls executed server-side, real post URLs returned with citations.

Proposed fix

1. Add xai as a first-class provider in crates/yoetz-cli/src/providers/mod.rs:

   • default_base_url("xai") => "https://api.x.ai/v1"
   • default_api_key_env("xai") => "XAI_API_KEY"

2. Add a Responses-API path for xAI (and any other provider that exposes the OpenAI-Responses-shaped endpoint with server-side tools). A new providers/xai.rs that maps messages to input and supports a tools array. Optionally generalise via a responses_endpoint: true flag on providers.

3. Add CLI flags on yoetz ask / yoetz council:

   • --x-search to inject {"type":"x_search"}
   • --web-search to inject {"type":"web_search"} (xAI also exposes this)
   • Optional --tool <name> for arbitrary server-side tools

4. Surface citations from the Responses API output (the annotations field) in --format json output so callers can post-process.

5. Registry update: mark x-ai/grok-4, x-ai/grok-4.3, x-ai/grok-4.20 as provider: xai (not openrouter) when --x-search is requested, OR fall back to OpenRouter when the tool flag is absent.

Why this matters

Live X search is a primary differentiator of Grok. Without it, yoetz ask against Grok is equivalent to any other commodity LLM call. Adding --x-search makes Grok useful for the OSINT / market-research workflows that yoetz is naturally suited for.

Repro context

Found while doing competitive recon on Hebrew posts about Israeli small-UAV licensing for an AskVirgil research note. The workaround surfaced 5 cited X posts in a single call.

[avivsinai]

Reviewed against current main bad57a5.

This is still a valid enhancement. Current code still routes x-ai/grok-* through OpenRouter registry resolution (ask / council / review paths), has no first-class xai provider path for x_search, and the existing /responses code is OpenAI-specific rather than a general xAI Agent Tools client.

Suggested implementation split:

• Add first-class xAI auth/base-url handling and a Responses API client that can pass server-side tools.
• Add explicit CLI surface for x_search / web search rather than relying on registry web_search metadata alone.
• Preserve OpenRouter as the default route when no xAI tool flag is requested.
• Add mocked provider tests for request shape, tool injection, error handling, and citation/annotation output in JSON.

Leaving open; no product decision needed before an implementation PR.