Add low VRAM mode, CPU-only mode + image pre-loading fix #407
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Low VRAM mode implemented: Introduced the unimplemented option lingering around in
streamlit_helpers.py
to run the model with half-precision (float16
). Allows efficient usage on GPUs with limited memory, lowering memory requirements by half. I've tested this on a RTX 3060 w/ 12GB VRAM.CPU-only mode: added an option to run the model on CPUs for users without the necessary CUDA GPU hardware, though at slower speeds.
Both can be set with True/False in
streamlit_helpers.py
, although a more viable solution in the future would be to i.e. add a separateconfig.ini
w/configparser
for the entire framework.video_sampling.py