fix: inject stream_options for usage data in streaming#346
fix: inject stream_options for usage data in streaming#346East-rayyy wants to merge 1 commit intodecolua:masterfrom
Conversation
Add stream_options: { include_usage: true } to OpenAI-format streaming
requests in BaseExecutor. This tells providers to include token usage
in the final streaming chunk, fixing token counts showing as 0.
Only injected when body.messages exists (OpenAI format) and the client
hasn't already set stream_options. Providers using non-OpenAI formats
(Gemini, Claude, Kiro) are unaffected since their bodies lack messages.
|
Thank you so much for identifying this issue and providing a solution! The stream_options injection is exactly what's needed to get usage data from streaming requests. I've implemented your fix, but applied it only to the iFlow executor instead of BaseExecutor. This gives us more control and allows us to test with iFlow first before rolling out to all providers globally. The change:
If this works well with iFlow, we can consider extending it to other providers in a future update. I apologize for the delay in reviewing – I've been extremely busy and couldn't get to it sooner. Thanks again for your great contribution! 🙏 Commit: e9ccae4 |
|
Closing this PR as the fix has been applied to iFlow executor (instead of BaseExecutor for more control). Thank you! |
Summary
stream_options: { include_usage: true }inBaseExecutor.transformRequestfor OpenAI-format streaming requests that don't already specify itsuper.transformRequest()Approach
The injection happens at the base executor level, which is the DRY solution since most providers use OpenAI-compatible format. Providers that don't support this field either silently ignore it (OpenRouter, Fireworks, Cerebras) or have custom executors that can strip it. The Codex executor already deletes
stream_optionsin its owntransformRequest, so no conflict there.Providers tested/verified:
Test plan
Fixes #74