You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is this a new feature, an improvement, or a change to existing functionality?
Improvement
How would you describe the priority of this feature request
High
Please provide a clear description of problem this feature solves
The first iteration of Sherlock nodes did not utilize MRC in order to support using coroutines in Morpheus before they are fully supported in MRC. For future compatibility, all Sherlock nodes should be converted to use MRC modules to take advantage of coroutine nodes when they are supported in MRC.
Describe your ideal solution
Sherlock nodes create a subgraph within the LLMEngine class. This subgraph can be converted to MRC nodes and connected via edges where each Sherlock nodes is a MRC node. This will allow for fully async pipelining of the LLMEngine (i.e. allowing simultaneous execution of two sibling nodes in the LLMEngine which is not possible now).
Completion Criteria
All 4 LLM examples run exactly as they did before without any code changes to the files in the examples directory (changes to nodes in morpheus/llm are ok)
Performance of the examples before the change and after the change match (within reason). See #sherlock per
All existing tests pass
The LLMEngine can be run both inside of a pipeline (where nodes will be converted to MRC modules) as well as outside of a pipeline (exactly as they are now)
This is to avoid requiring an entire pipeline for debugging and testing of the LLMEngine
Additional context
No response
Code of Conduct
I agree to follow this project's Code of Conduct
I have searched the open feature requests and have found no duplicates for this feature request
The text was updated successfully, but these errors were encountered:
Is this a new feature, an improvement, or a change to existing functionality?
Improvement
How would you describe the priority of this feature request
High
Please provide a clear description of problem this feature solves
The first iteration of Sherlock nodes did not utilize MRC in order to support using coroutines in Morpheus before they are fully supported in MRC. For future compatibility, all Sherlock nodes should be converted to use MRC modules to take advantage of coroutine nodes when they are supported in MRC.
Describe your ideal solution
Sherlock nodes create a subgraph within the
LLMEngine
class. This subgraph can be converted to MRC nodes and connected via edges where each Sherlock nodes is a MRC node. This will allow for fully async pipelining of theLLMEngine
(i.e. allowing simultaneous execution of two sibling nodes in theLLMEngine
which is not possible now).Completion Criteria
morpheus/llm
are ok)LLMEngine
can be run both inside of a pipeline (where nodes will be converted to MRC modules) as well as outside of a pipeline (exactly as they are now)LLMEngine
Additional context
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: