The local API proxy forwards chat-completion requests to llama-server with a reqwest client built using a total-request timeout:
// src-tauri/src/core/server/proxy.rs (both the general and no_proxy local clients)
let client = Client::builder()
.timeout(std::time::Duration::from_secs(proxy_timeout)) // default 600s
.build()?;
reqwest::ClientBuilder::timeout() caps the entire request, including reading the streamed response body. For a streaming generation that means the timeout budget covers prompt prefill + the whole decode. On a slow-prefill model with a large context (e.g. the ternary 27B after a long agentic thread), a single healthy turn can approach or exceed the cap and get cut mid-stream, surfacing to the user as a generation error / unexpected EOF even though the server is fine and still working.
The correct semantic for a streaming proxy is an idle / read timeout (abort only if no bytes arrive for N seconds), not a total-duration cap. That way a long generation with steady token output never trips it, but a genuinely hung server still does.
reqwest::ClientBuilder::read_timeout() provides exactly this, but it was added in reqwest 0.12; this crate is on 0.11.27, so the fix requires a dependency upgrade.
Scope / plan:
- Upgrade
reqwest 0.11 → 0.12 (note blast radius: reqwest is used across src-tauri and pulled in by the MCP client via the transport-*-reqwest features, so the bump likely cascades to rmcp and related deps).
- Switch the two proxy forwarding clients (
proxy.rs) to .read_timeout(...) for the streaming path, keeping a total .timeout() only where a hard cap is actually wanted (non-streaming calls).
- Verify a long local generation no longer gets cut, and that a killed
llama-server still fails promptly.
Tracked separately from the partial-work-preservation fix (which addresses losing output when a generation does get interrupted, regardless of cause).
The local API proxy forwards chat-completion requests to
llama-serverwith a reqwest client built using a total-request timeout:reqwest::ClientBuilder::timeout()caps the entire request, including reading the streamed response body. For a streaming generation that means the timeout budget covers prompt prefill + the whole decode. On a slow-prefill model with a large context (e.g. the ternary 27B after a long agentic thread), a single healthy turn can approach or exceed the cap and get cut mid-stream, surfacing to the user as a generation error /unexpected EOFeven though the server is fine and still working.The correct semantic for a streaming proxy is an idle / read timeout (abort only if no bytes arrive for N seconds), not a total-duration cap. That way a long generation with steady token output never trips it, but a genuinely hung server still does.
reqwest::ClientBuilder::read_timeout()provides exactly this, but it was added in reqwest 0.12; this crate is on 0.11.27, so the fix requires a dependency upgrade.Scope / plan:
reqwest0.11 → 0.12 (note blast radius: reqwest is used acrosssrc-tauriand pulled in by the MCP client via thetransport-*-reqwestfeatures, so the bump likely cascades tormcpand related deps).proxy.rs) to.read_timeout(...)for the streaming path, keeping a total.timeout()only where a hard cap is actually wanted (non-streaming calls).llama-serverstill fails promptly.Tracked separately from the partial-work-preservation fix (which addresses losing output when a generation does get interrupted, regardless of cause).