Skip to content

fix: inject stream_options for usage data in streaming#346

Closed
East-rayyy wants to merge 1 commit intodecolua:masterfrom
East-rayyy:fix/inject-stream-usage-options
Closed

fix: inject stream_options for usage data in streaming#346
East-rayyy wants to merge 1 commit intodecolua:masterfrom
East-rayyy:fix/inject-stream-usage-options

Conversation

@East-rayyy
Copy link
Contributor

Summary

  • Injects stream_options: { include_usage: true } in BaseExecutor.transformRequest for OpenAI-format streaming requests that don't already specify it
  • Ensures iFlow executor inherits the injection by calling super.transformRequest()
  • This tells compatible providers (OpenAI, Azure, Gemini, Ollama, Together AI, etc.) to include token usage data in the final streaming chunk, fixing token counts showing as 0

Approach

The injection happens at the base executor level, which is the DRY solution since most providers use OpenAI-compatible format. Providers that don't support this field either silently ignore it (OpenRouter, Fireworks, Cerebras) or have custom executors that can strip it. The Codex executor already deletes stream_options in its own transformRequest, so no conflict there.

Providers tested/verified:

  • Safe to send: OpenAI, Azure OpenAI, Gemini, Ollama, Together AI
  • Silently ignored: OpenRouter, Fireworks, Cerebras, DeepInfra
  • Already handled: Codex executor strips it, Anthropic/Claude uses different format

Test plan

  • Verify streaming responses from OpenAI-compatible providers now include usage data in the final chunk
  • Verify Codex streaming still works (its executor already deletes stream_options)
  • Verify iFlow streaming works with the super call
  • Verify providers that ignore unknown fields still respond correctly

Fixes #74

Add stream_options: { include_usage: true } to OpenAI-format streaming
requests in BaseExecutor. This tells providers to include token usage
in the final streaming chunk, fixing token counts showing as 0.

Only injected when body.messages exists (OpenAI format) and the client
hasn't already set stream_options. Providers using non-OpenAI formats
(Gemini, Claude, Kiro) are unaffected since their bodies lack messages.
@decolua
Copy link
Owner

decolua commented Mar 23, 2026

Thank you so much for identifying this issue and providing a solution! The stream_options injection is exactly what's needed to get usage data from streaming requests.

I've implemented your fix, but applied it only to the iFlow executor instead of BaseExecutor. This gives us more control and allows us to test with iFlow first before rolling out to all providers globally.

The change:

  • ✅ Injects stream_options: { include_usage: true } for iFlow streaming requests
  • ✅ Only when body.messages exists (OpenAI format)
  • ✅ Only when client hasn't already set stream_options
  • ✅ Fixes token counts showing as 0 for iFlow

If this works well with iFlow, we can consider extending it to other providers in a future update.

I apologize for the delay in reviewing – I've been extremely busy and couldn't get to it sooner.

Thanks again for your great contribution! 🙏

Commit: e9ccae4

@decolua
Copy link
Owner

decolua commented Mar 23, 2026

Closing this PR as the fix has been applied to iFlow executor (instead of BaseExecutor for more control). Thank you!

@decolua decolua closed this Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Input/Output token counts always show 0 for all requests (recent regression)

2 participants