Skip to content
This repository has been archived by the owner on Dec 19, 2024. It is now read-only.

Reduce memory footprint for captures.py using Dask Bag #91

Open
adason opened this issue Sep 12, 2020 · 0 comments
Open

Reduce memory footprint for captures.py using Dask Bag #91

adason opened this issue Sep 12, 2020 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@adason
Copy link
Contributor

adason commented Sep 12, 2020

Why you need this feature:

The current implementation of captures.py loads all annotations for all captures frames into memory. This can potentially create memory hog as annotations can greatly increase memory usage.

Describe the solution you'd like:

Explore using Dask Bag to filter annotations that does not need to be loaded into memory. They should only be loaded when Dataset.__getitem__() is called to generate training examples.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

@adason adason added the enhancement New feature or request label Sep 12, 2020
@adason adason self-assigned this Sep 12, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant