Create Sherlock example for Completion pipeline

[mdemoret-nv]

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

High

Please provide a clear description of problem this feature solves

As part of the Sherlock work, an example showing how to use Morpheus to execute multiple LLM completion queries inside of a pipeline.

Describe your ideal solution

Purpose

The purpose of this example is to illustrate how a user could build a pipeline while will integrate an LLM service into a Morpheus pipeline. This example is the most basic building block on how to use the LLMEngine without requiring any external services or pre-existing data.

Scenario

This example will show one single implementation but the pipeline and components could be used in many scenarios with different requirements. At a high level, the following illustrates different customization points for this pipeline and the specific choices made for this example:

• LLM Service
  • This pipeline could support any type of LLM service which is compatible with our LLMService interface.
  • Such services include OpenAI, NeMo, or even running locally using llama-cpp-python
  • For this example, we will focus on using NeMo as the LLM service. This provides an opportunity to show the benefits of using NeMo over other LLM services and helps build components around the NeMo ecosystem. Additionally, more complex pipelines could be built using NeMo + Inform without requiring any changes to the pipeline.
• Downstream tasks
  • After the LLM has been run, the output of the model could be used in any number of tasks such as training a model, running analysis, or even simulating an attack.
  • For this example, we will not have any downstream tasks to keep the implementation simple and the focus on the LLMEngine

Implementation

This example will be composed of a single click command.

Morpheus pipeline

The Morpheus pipeline is built using the following components:

1. An InMemorySourceStage to hold the LLM queries in a DataFrame
2. A DeserializationStage to convert the MessageMeta objects into ControlMessages needed by the LLMEngine
3. New functionality was added to the DeserializeStage to support ControlMessages and add a default task to each message.
4. A LLMEngineStage then wraps the core LLMEngine functionality
   1. An ExtracterNode pulls the questions out of the DataFrame
   2. A PromptTemplateNode converts the data and a template into the final inputs for the LLM
   3. The LLM is executed using an LLMGenerateNode to run the LLM queries
   4. Finally, the responses are put back into the ControlMessage using a SimpleTaskHandler
5. The pipeline concludes with an InMemorySink stage to store the results.

Completion Criteria

The following items need to be satisfied to consider this issue complete:

• A README.md containing information on the following:
  • Background information about the problem at hand
  • Information about the specific implementation and reasoning behind design decisions (use content from this issue)
  • Step-by-step instructions for running the following:
    • How to run the Morpheus pipeline
      • Including instructions on using the NeMo service
  • The README.md should be linked in the developer docs
• A functioning Morpheus pipeline command which satisfies the following:
  • Should run without error using all default arguments
  • Correctly run the queries through the specified NeMo model
  • Provide information about the success or failure of the pipeline. Including the number of queries run, throughput and total runtime.
• Tests should be added which include the following
  • Test successfully running the Morpheus pipeline
    • Outputs from the model can be pre-calculated using the NeMo playground or the NeMo curl API to verify the responses from Morpheus are correct

Dependent Issues

The following issues should be resolved before this can be completed:

Tasks

Beta Give feedback

1. [FEA]: Create LLM Engine Core Functionality #1178
   feature request sherlock +2
2. [FEA]: Create a NeMo Service and NeMo Stage #1130
   feature request sherlock +2
   
3. [FEA]: Create Templating Prompt Generator for the LLM Engine #1179
   feature request sherlock +2
   
4. Add documentation and tests for the ExtractorNode #1277
   4 of 4
   sherlock +1
   
5. Add documentation and tests for the LLMEngineStage #1278
   4 of 4
   sherlock +1
   
6. Add documentation and tests for the LLMGenerateNode #1279
   4 of 4
   sherlock +1
   
7. Add documentation and tests for the PromptTemplateNode #1282
   4 of 4
   sherlock +1
   
8. Add documentation and tests for the SimpleTaskHandler #1302
   4 of 4
   sherlock +1
   
9. Add documentation and tests for the LLMService #1303
   4 of 4
   sherlock +1
   

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct
• I have searched the open feature requests and have found no duplicates for this feature request

[mdemoret-nv]

Closing since it was completed in 23.11