You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
tensorrtllm backend doesn't work for us because of this bug: #598. So I have to use python backend. However, it only supports detached model which we don't need.
System Info
tensorrtllm backend doesn't work for us because of this bug: #598. So I have to use python backend. However, it only supports detached model which we don't need.
Can we add support for non-detached mode?
Who can help?
@ncomly-nvidia
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
N/A
Expected behavior
N/A
actual behavior
N/A
additional notes
N/A
The text was updated successfully, but these errors were encountered: