class TritonPythonModel usage

[hoohooprog]

Hi

i've been thinking and researching on using TIS for a while now. I wanted to be slightly more thorough with my understanding of data flow rather than plug-and-play. I believe even plug-and-play requires a slight bit of SE networking skills.

but anyways I can't seem to locate where or how the very important class TritonPythonModel is used.

I found some other sub classes being used in the model.py file like class InferenceResponse from /core/python/tritonserver/_api/_response.

maybe i'm going about the wrong way to try to deploy a vLLM model, I would like to deploy and test Nvidia mistral Nemo

[hoohooprog]

or can anyone suggest a preferred way to test? breakpoints? print statements? thanks, I need some SE tooling refresher.

[hoohooprog]

this is the example tutorial that's serving as my template
https://awslabs.github.io/data-on-eks/docs/gen-ai/inference/GPUs/vLLM-NVIDIATritonServer#mistralaimistral-7b-instruct-v02

but i found some other tutorials that require to install backend, inferenceserverclient etc.
like this building from source https://github.com/triton-inference-server/python_backend?tab=readme-ov-file#building-from-source

it seems like there are a handful of ways of building the app (helm, yaml) that can obfuscate the true build process