Skip to content

Issues: ray-project/llmperf

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

20 Open 2 Closed
20 Open 2 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

How to use llmperf to test ollama performance (TTFT, etc)
#58 opened Jun 21, 2024 by alexhegit
Subsequent requests cannot be sent until 'num_concurrent_requests' requests have all finished
#56 opened Jun 18, 2024 by llsj14
2
Divide by zero: request_metrics[common_metrics.REQ_OUTPUT_THROUGHPUT] = num_output_tokens / request_metrics[common_metrics.E2E_LAT]
#55 opened Jun 13, 2024 by yaronr
Sagemaker client issue
#53 opened Jun 7, 2024 by SuchethaChintha
error in request_metrics dictionary implementation
#49 opened May 14, 2024 by Durga2Dash
Vertex AI API needs to be updated.
#48 opened May 14, 2024 by Durga2Dash
Since it does not have a 'setup.py' nor a 'setup.cfg', it cannot be installed in editable mode PEP 660
#47 opened May 9, 2024 by biubiu3721
unable to get the benchmark result
#44 opened Apr 4, 2024 by dipshirajput
1
Blocking on pending requests despite block == false
#43 opened Apr 3, 2024 by dacorvo
1
HUGGINGFACE set
#38 opened Mar 12, 2024 by capyun
bug of counting output tokens
#35 opened Feb 22, 2024 by irasin
llmperf not working for concurrent users
#34 opened Feb 13, 2024 by nkanike07
Bug: Hugging Face TGI not working
#33 opened Feb 7, 2024 by ptrmayer
Concurrency level is not handled properly
#32 opened Jan 26, 2024 by alexeykudinkin
Add memory bandwidth utilization metric
#31 opened Jan 25, 2024 by mmcclean-aws
Handling custom codes arguments like trust_remote_code
#30 opened Jan 11, 2024 by Akash08naik
usage for local models .
#29 opened Jan 11, 2024 by Akash08naik
Basic usage issue
#24 opened Dec 25, 2023 by wangxingjun778
litellm serializable issue ?
#21 opened Dec 6, 2023 by ishaan-jaff
Crashing with large number of concurrent users
#17 opened Nov 30, 2023 by francescov1
ProTip! Follow long discussions with comments:>50.