Skip to content

Commit

Permalink
fix: skip first token for azure openai async + stream
Browse files Browse the repository at this point in the history
  • Loading branch information
samuelrince committed Aug 23, 2024
1 parent 905db3f commit 9fcee1a
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions ecologits/tracers/openai_tracer.py
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,8 @@ async def openai_async_chat_wrapper_stream(
i = 0
token_count = 0
async for chunk in stream:
if i == 0 and chunk.model == "":
continue
if i > 0 and chunk.choices[0].finish_reason is None:
token_count += 1
request_latency = time.perf_counter() - timer_start
Expand Down

0 comments on commit 9fcee1a

Please sign in to comment.