-
Notifications
You must be signed in to change notification settings - Fork 125
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for a CPU-only Mode #1851
Draft
dagardner-nv
wants to merge
212
commits into
nv-morpheus:branch-24.10
Choose a base branch
from
dagardner-nv:david-cpu-only-mode-1846
base: branch-24.10
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Add support for a CPU-only Mode #1851
dagardner-nv
wants to merge
212
commits into
nv-morpheus:branch-24.10
from
dagardner-nv:david-cpu-only-mode-1846
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a fixup to commit 4e3edb9 that refactored the setup files Signed-off-by: Anuradha Karuppiah <[email protected]>
…avid-cpu-only-poc [no ci]
…pheus into david-cpu-only-poc
…rceStage to emit ControlMessages
…avid-cpu-only-mode-1846
…, allowing for a subclass without a Python impl
…avid-cpu-only-mode-1846-2410
rapids-bot bot
pushed a commit
to nv-morpheus/MRC
that referenced
this pull request
Sep 11, 2024
* Since CPU-only mode will become a supported feature we want to avoid unnecessary warnings. Relates to nv-morpheus/Morpheus#1851 Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #497
…der' it will always find a mode named 'dfencode', however this test doesn't provide much anyways
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
breaking
Breaking change
DO NOT MERGE
PR should not be merged; see PR for details
feature request
New feature or request
skip-ci
Optionally Skip CI for this PR
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
morpheus.config.ExecutionMode
with membersGPU
&CPU
along with a newmorpheus.config.Config.execution_mode
attribute.Config.execution_mode
will always default toGPU
supported_execution_modes
toStageBase
which returnsExecutionMode.GPU
by default. This ensures that building a pipeline with a stage not matching the execution mode will raise a reasonable error to the user.CpuOnlyMixin
andGpuAndCpuMixin
mixins to automate overriding this, and makes it easier for users to determine which execution modes a given stage supports at a glance.cudf
or any other GPU based package which will fail on import at the top-level of a module. This is important for stage, message and modules which are automatically imported by the morpheus CLI tool.morpheus.utils.type_utils
(ex:get_df_pkg
,is_cudf_type
) to help avoid importing cudf directlyConfig.freeze
method which will make a config object immutable. This will be called the first time a config object is used to construct a pipeline or stage object. Prevents the possibility of config parameters from being changed in the middle of pipeline construction.CudfHelper::load
is no longer called automatically on import, instead it is called manually on pipeline build when execution mode is GPU.ControlMessage
CPU_ONLY
environment variable is defineddocker/run_container_dev.sh
will launch the container using therunc
runtime.gpu_and_cpu_mode
test marker to explicitly indicate a test intended to be parameterized over execution modes.ControlMessage
AppShieldSourceStage
now emitsControlMessage
s,AppShieldMessageMeta
is now deprecatedAutoencoderSourceStage
and thusAzureSourceStage
,CloudTrailSourceStage
, andDuoSourceStage
now emitControlMessage
,UserMessageMeta
is now deprecated.Known Issues:
Stages that support GPU & CPU mode:
Closes #1646
Closes #1846
Closes #1852
Open Questions:
PreprocessFILStage
be removed, given that the C++ impl is the only one that will be used?CppConfig
be removed/renamed? The concept of a C++ mode is becoming a bit confusing given that we want to support LLM pipelines in CPU/Python mode, yet much of that code only contains a C++ impl. The big advantage that the current singleton has is that message constructors have access to that to determine if the C++ or Python impl should be used.MessageMeta
which can store arbitrary Python objects in a map, and then handle the API in the Python bindings.By Submitting this PR I confirm: