We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
其中perf_analyzer命令如下: perf_analyzer -m tensorrt_llm --concurrency-range 1:5 -u localhost:8004 --shape bad_words_list:2,1 --shape draft_input_ids:1 --shape embedding_bias:1 --shape input_ids:1 --shape lora_config:1,3 --shape lora_weights:1,3 --shape prompt_embedding_table:1,1 --shape stop_words_list:2,1 -i grpc --streaming --percentile=95 --async
报错截图如下:
怎么解决?
The text was updated successfully, but these errors were encountered:
尝试指定一个request_id?
Sorry, something went wrong.
没找到怎么指定,perf_analyzer本身没有这个参数
老哥,你是在用这个测trtllm的吞吐量吗,这个命令有效果吗,还有什么办法可以测试最大吞吐量
No branches or pull requests
其中perf_analyzer命令如下:
perf_analyzer -m tensorrt_llm --concurrency-range 1:5 -u localhost:8004 --shape bad_words_list:2,1 --shape draft_input_ids:1 --shape embedding_bias:1 --shape input_ids:1 --shape lora_config:1,3 --shape lora_weights:1,3 --shape prompt_embedding_table:1,1 --shape stop_words_list:2,1 -i grpc --streaming --percentile=95 --async
报错截图如下:
怎么解决?
The text was updated successfully, but these errors were encountered: