Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

High

Please provide a clear description of problem this feature solves

This feature would allow Morpheus pipelines to integrate with NVIDIA's LLM service, NeMo, by sending inference requests to the service from a stage in the pipeline.

The ability to run LLM models inside of a Morpheus pipeline will allow for pipelines to execute complex NLP tasks with very large models. Often these models would be too large to run inside of a Morpheus pipeline so sending the requests off to an external service fits well with other inference services like Triton.

Describe your ideal solution

This new feature should be built from 2 components:

1. A NeMo LLM service which lives outside of the pipeline
   1. The LLM service should live outside of the pipeline to allow multiple stages to utilize the same LLM model, but batch their requests together. The best way to do this is make a singleton service which can be accessed at any time from multiple stages.
   2. Requests sent to this service should be batched and then sent off to the NeMo endpoint via the nemo_llm library (python) or CURL (C++)
2. A NeMo LLM Inference Stage which lives inside of the pipeline
   1. This stage will primarily interact with the LLM service, sending input messages to the LLM service for inference
   2. Returned messages from the LLM service will be sent to the next stage in the pipeline.

Configurable Options
The NeMo Inference stage should include (but not be limited to the following configurable parameters:

• The column to use as the text to use for the inference request
• The model name
• The model customization ID
• NeMo endpoint
• API Key
• Organization Key
• Any model parameters that nemo_llm supports
  • For example, tokens_to_generate, stop, temperature, etc.

Describe any alternatives you have considered

A test prototype has been created here: https://github.com/mdemoret-nv/Morpheus/tree/mdd_nemo-stage/examples/nemo

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct
• I have searched the open feature requests and have found no duplicates for this feature request