Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

当我用perf_analyzer测试性能时,出现“Thread [0] had error: Cannot send stop request without specifying a request_id”错误 #112

Open
MuyeMikeZhang opened this issue Apr 28, 2024 · 3 comments

Comments

@MuyeMikeZhang
Copy link

其中perf_analyzer命令如下:
perf_analyzer -m tensorrt_llm --concurrency-range 1:5 -u localhost:8004 --shape bad_words_list:2,1 --shape draft_input_ids:1 --shape embedding_bias:1 --shape input_ids:1 --shape lora_config:1,3 --shape lora_weights:1,3 --shape prompt_embedding_table:1,1 --shape stop_words_list:2,1 -i grpc --streaming --percentile=95 --async

报错截图如下:
image

怎么解决?

@Tlntin
Copy link
Owner

Tlntin commented Apr 28, 2024

尝试指定一个request_id?

@MuyeMikeZhang
Copy link
Author

没找到怎么指定,perf_analyzer本身没有这个参数

@lzcchl
Copy link

lzcchl commented May 7, 2024

老哥,你是在用这个测trtllm的吞吐量吗,这个命令有效果吗,还有什么办法可以测试最大吞吐量

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants