Will the current backend of Triton Inference Server will be supported in Dynamo ?

[MatthieuToulemont]

Hello Nvidia team, and thank you for the great work,

We use Triton Inference Server a lot in my company but it seems that Dynamo addresses some of the shortcomings of Triton (multi node inference for instance).

I am wondering if the backends currently supported in Triton will be integrated in Dynamo ? We do a lot of computer vision and so far TensorRT LLM is not required to run most of the computer vision flows yet they would still benefit from multi node inference.

Very curious about your point of view

[dzier]

@MatthieuToulemont , we are still in the early stages of adding Triton Backend support into Dynamo, and do not have a solid road map yet on when that support will be available.

Out of curiosity, how do you envision the benefit of multi-node inference on computer vision flows? Do see individual models sharded across multiple nodes or ensembles that have different models/stages on different nodes?