feat(providers): add openai-oxide as alternative OpenAI provider#521
feat(providers): add openai-oxide as alternative OpenAI provider#521fortunto2 wants to merge 3 commits intomoltis-org:mainfrom
Conversation
…cket Add openai-oxide 0.11.1 as alternative OpenAI provider supporting: - Chat Completions and Responses API (WireApi config) - SSE, WebSocket, and Auto transport modes - Full tool calling (extraction, streaming, replay) - Vision, reasoning effort, model discovery - 888 lines replacing 5300+ lines of manual HTTP/SSE code Replaces PR moltis-org#487 (rebased on current main, oxide bumped 0.10.1 → 0.11.1).
Greptile SummaryThis PR introduces Two P1 bugs need resolution before merge:
Additionally, Confidence Score: 4/5Not safe to merge as-is — tool results are silently dropped in the Responses API path, which breaks agent loops using tool calling. Two P1 issues block merge: the
Important Files Changed
Sequence DiagramsequenceDiagram
participant C as Caller
participant P as OpenAiOxideProvider
participant OAI as openai-oxide client
C->>P: stream_with_tools(messages, tools)
alt WireApi::ChatCompletions
P->>P: build_chat_messages()
P->>OAI: chat().completions().create_stream()
OAI-->>P: SSE chunks (content / tool_calls / usage)
P-->>C: StreamEvent::Delta / ToolCallStart / ToolCallArgumentsDelta / ToolCallComplete / Done
else WireApi::Responses + SSE
P->>P: split_for_responses() ⚠️ drops Tool & Multimodal
P->>P: build_responses_request()
P->>OAI: responses().create_stream()
OAI-->>P: ResponseStreamEvent
P->>P: process_response_event()
P-->>C: StreamEvent::Delta / ToolCallStart / ToolCallArgumentsDelta / ToolCallComplete / Done
else WireApi::Responses + WebSocket
P->>OAI: ws_session()
OAI-->>P: WsSession
P->>OAI: session.send_stream()
OAI-->>P: ResponseStreamEvent (WS)
P->>P: process_response_event()
P-->>C: StreamEvent::Delta / ... / Done
else WireApi::Responses + Auto
P->>OAI: ws_session() [attempt]
alt WS succeeds
OAI-->>P: WsSession
P->>OAI: session.send_stream()
else WS fails
P->>P: fallback to stream_responses (SSE)
end
P-->>C: StreamEvent (from whichever path succeeded)
end
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| #[async_trait] | ||
| impl LlmProvider for OpenAiOxideProvider { | ||
| fn name(&self) -> &str { | ||
| self.alias.as_deref().unwrap_or("openai-oxide") | ||
| } | ||
|
|
||
| fn id(&self) -> &str { | ||
| &self.model | ||
| } | ||
|
|
||
| fn supports_tools(&self) -> bool { | ||
| true | ||
| } | ||
|
|
||
| fn supports_vision(&self) -> bool { | ||
| true | ||
| } | ||
|
|
||
| fn reasoning_effort(&self) -> Option<ReasoningEffort> { | ||
| self.reasoning_effort | ||
| } | ||
|
|
||
| fn with_reasoning_effort( | ||
| self: Arc<Self>, | ||
| effort: ReasoningEffort, | ||
| ) -> Option<Arc<dyn LlmProvider>> { | ||
| Some(Arc::new(Self { | ||
| model: self.model.clone(), | ||
| client: self.client.clone(), | ||
| alias: self.alias.clone(), | ||
| reasoning_effort: Some(effort), | ||
| wire_api: self.wire_api, | ||
| stream_transport: self.stream_transport, | ||
| })) | ||
| } | ||
|
|
||
| fn context_window(&self) -> u32 { | ||
| 128_000 |
There was a problem hiding this comment.
split_for_responses handles System, User { Text }, and Assistant { content } but the catch-all _ => {} silently discards both ChatMessage::Tool { ... } (tool-call results) and ChatMessage::User { content: UserContent::Multimodal(_) } (vision messages).
For any agent turn that involves tool calling with WireApi::Responses, the ChatMessage::Tool result messages produced by the tool executor will be dropped before the request is built. The resulting ResponseCreateRequest will contain an assistant message that references a function call with no corresponding output item, which causes the Responses API to return an error or silently loop — breaking the agent loop entirely.
The Responses API represents tool results as function_call_output input items. Something like:
ChatMessage::Tool { tool_call_id, content, .. } => {
input.push(ResponseInputItem {
role: openai_oxide::types::responses::Role::Tool,
content: serde_json::json!({
"type": "function_call_output",
"call_id": tool_call_id,
"output": content,
}),
});
}Vision/multimodal messages (UserContent::Multimodal) also need handling — despite the PR claiming vision support, it only works via Chat Completions, not Responses API.
| ); | ||
| } | ||
|
|
||
| #[cfg(feature = "provider-openai-oxide")] | ||
| fn register_openai_oxide_providers( | ||
| &mut self, | ||
| config: &ProvidersConfig, | ||
| env_overrides: &HashMap<String, String>, | ||
| ) { | ||
| if !config.is_enabled("openai") { | ||
| return; | ||
| } | ||
|
|
||
| let Some(key) = resolve_api_key(config, "openai", "OPENAI_API_KEY", env_overrides) else { | ||
| return; | ||
| }; | ||
|
|
||
| let base_url = config | ||
| .get("openai") | ||
| .and_then(|e| e.base_url.clone()) | ||
| .or_else(|| env_value(env_overrides, "OPENAI_BASE_URL")) | ||
| .unwrap_or_else(|| "https://api.openai.com/v1".into()); | ||
|
|
||
| let model_id = configured_models_for_provider(config, "openai") | ||
| .into_iter() | ||
| .next() | ||
| .unwrap_or_else(|| "gpt-4o".to_string()); | ||
|
|
||
| let alias = config.get("openai").and_then(|e| e.alias.clone()); | ||
| let provider_label = alias.clone().unwrap_or_else(|| "openai-oxide".into()); | ||
| if self.has_model_any_provider(&model_id) { | ||
| return; | ||
| } | ||
|
|
||
| let provider = Arc::new(openai_oxide_provider::OpenAiOxideProvider::with_alias( | ||
| key, | ||
| model_id.clone(), | ||
| base_url, | ||
| alias, | ||
| )); | ||
| self.register( | ||
| ModelInfo { | ||
| id: model_id, | ||
| provider: provider_label, | ||
| display_name: "GPT-4o (openai-oxide)".into(), | ||
| created_at: None, | ||
| }, | ||
| provider, | ||
| ); | ||
| } | ||
|
|
||
| #[cfg(feature = "provider-openai-codex")] | ||
| fn register_openai_codex_providers( |
There was a problem hiding this comment.
display_name hardcoded regardless of model; only first configured model registered
Two related issues in register_openai_oxide_providers:
1. Hardcoded display name: The display_name is unconditionally set to "GPT-4o (openai-oxide)" regardless of which model is actually selected. If a user sets model = "gpt-4-turbo" or model = "o3-mini" in their config, the UI will still show "GPT-4o (openai-oxide)", surfacing incorrect data.
// Current (wrong):
display_name: "GPT-4o (openai-oxide)".into(),
// Should be derived from the actual model_id:
display_name: format!("{model_id} (openai-oxide)"),2. Only the first configured model is registered: configured_models_for_provider(config, "openai").into_iter().next() silently discards every configured model after the first. Other providers (e.g. async-openai) register all models; here a user who has configured ["gpt-4o", "gpt-4-turbo"] will only get gpt-4o registered, with no indication the others were ignored.
| request.stream_options = Some(StreamOptions { include_usage: Some(true) }); | ||
| stream_chat(&self.client, request) | ||
| } | ||
| WireApi::Responses => { | ||
| let request = self.build_responses_request(&messages, &tools); | ||
| match self.stream_transport { | ||
| ProviderStreamTransport::Websocket => { | ||
| stream_responses_ws(&self.client, request) | ||
| } | ||
| ProviderStreamTransport::Auto => { | ||
| // Auto: try WS, fallback to SSE | ||
| stream_responses_auto(&self.client, request) | ||
| } | ||
| ProviderStreamTransport::Sse => { | ||
| stream_responses(&self.client, request) | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl OpenAiOxideProvider { | ||
| fn build_responses_request( | ||
| &self, | ||
| messages: &[ChatMessage], | ||
| tools: &[serde_json::Value], | ||
| ) -> ResponseCreateRequest { | ||
| let (instructions, input) = split_for_responses(messages); | ||
| let mut request = ResponseCreateRequest::new(&self.model) | ||
| .input(ResponseInput::Messages(input)); | ||
| if let Some(instr) = instructions { | ||
| request = request.instructions(instr); | ||
| } | ||
| if !tools.is_empty() { | ||
| request.tools = Some(tools_to_response_tools(tools)); |
There was a problem hiding this comment.
ToolCallComplete events emitted in non-deterministic order
In stream_chat, ToolCallComplete events for multiple simultaneous tool calls are emitted by iterating seen_starts.keys() — a HashMap<i32, bool> — whose iteration order is not guaranteed:
for &idx in seen_starts.keys() {
yield StreamEvent::ToolCallComplete { index: idx as usize };
}If a downstream consumer expects ToolCallComplete(n) to follow ToolCallStart(n) in index order (which is a reasonable assumption), this can produce out-of-order events when n > 1. Consider sorting the keys before iterating:
let mut sorted_keys: Vec<i32> = seen_starts.keys().copied().collect();
sorted_keys.sort_unstable();
for idx in sorted_keys {
yield StreamEvent::ToolCallComplete { index: idx as usize };
}…based switching - Register oxide under its own config key `[providers.openai-oxide]` - Both oxide and async-openai compile by default (no feature flag switching) - Oxide discovers all OpenAI models (same catalog as built-in) - Reads wire_api and stream_transport from provider config - Switch providers by renaming config section, no rebuild needed
… ordering - Map ChatMessage::Tool as function_call_output in Responses API path - Map UserContent::Multimodal (flatten text) in Responses API path - Replace HashMap with BTreeMap for deterministic ToolCallComplete ordering
1 participant
Summary
openai-oxide0.11.1 as standalone OpenAI provider ([providers.openai-oxide])wire_apiconfigprovider-async-openaiandprovider-openai-oxidecompile by defaultConfig-based switching
Use oxide (Responses API + WebSocket):
Use built-in (Chat Completions + SSE):
Feature comparison
Validation
Completed
cargo checkpasses (full workspace)cargo buildpassesmoltis doctor— provider detectedRemaining
cargo test(provider unit tests)just lint/just format-checkManual QA
[providers.openai-oxide]to config withwire_api = "responses"OPENAI_API_KEY, start gatewayopenai-oxide::model in UI[providers.openai]— verify built-in works🤖 Generated with Claude Code