Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Response time per N tokens #19

Open
maralski opened this issue Aug 8, 2023 · 0 comments
Open

Response time per N tokens #19

maralski opened this issue Aug 8, 2023 · 0 comments
Labels
enhancement Feature Improvements

Comments

@maralski
Copy link

maralski commented Aug 8, 2023

Dashboard should display requests per N token.

Summary

Current dashboard displays average response time. The average will fluctuate as it dependents on token count.

Desired Behavior

A more stable response time metric is one that is normalised by token count.

Possible Solution

Add a billboard and time series widgets that reports response time per 100 tokens.
@maralski maralski added the enhancement Feature Improvements label Aug 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement Feature Improvements
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@maralski