Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA]: Convert Sherlock nodes to modules #1425

Open
2 of 7 tasks
mdemoret-nv opened this issue Dec 12, 2023 · 0 comments
Open
2 of 7 tasks

[FEA]: Convert Sherlock nodes to modules #1425

mdemoret-nv opened this issue Dec 12, 2023 · 0 comments
Assignees
Labels
feature request New feature or request sherlock Issues/PRs related to Sherlock workflows and components

Comments

@mdemoret-nv
Copy link
Contributor

Is this a new feature, an improvement, or a change to existing functionality?

Improvement

How would you describe the priority of this feature request

High

Please provide a clear description of problem this feature solves

The first iteration of Sherlock nodes did not utilize MRC in order to support using coroutines in Morpheus before they are fully supported in MRC. For future compatibility, all Sherlock nodes should be converted to use MRC modules to take advantage of coroutine nodes when they are supported in MRC.

Describe your ideal solution

Sherlock nodes create a subgraph within the LLMEngine class. This subgraph can be converted to MRC nodes and connected via edges where each Sherlock nodes is a MRC node. This will allow for fully async pipelining of the LLMEngine (i.e. allowing simultaneous execution of two sibling nodes in the LLMEngine which is not possible now).

Completion Criteria

  • All 4 LLM examples run exactly as they did before without any code changes to the files in the examples directory (changes to nodes in morpheus/llm are ok)
  • Performance of the examples before the change and after the change match (within reason). See #sherlock per
  • All existing tests pass
  • The LLMEngine can be run both inside of a pipeline (where nodes will be converted to MRC modules) as well as outside of a pipeline (exactly as they are now)
    • This is to avoid requiring an entire pipeline for debugging and testing of the LLMEngine

Additional context

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
  • I have searched the open feature requests and have found no duplicates for this feature request
@mdemoret-nv mdemoret-nv added feature request New feature or request sherlock Issues/PRs related to Sherlock workflows and components labels Dec 12, 2023
@mdemoret-nv mdemoret-nv removed this from the 24.03 - Sherlock Improvements milestone Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request sherlock Issues/PRs related to Sherlock workflows and components
Projects
Status: In Progress
Development

No branches or pull requests

2 participants