Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata: Scan only what's needed #201

Open
Tracked by #197
tasket opened this issue May 30, 2024 · 0 comments
Open
Tracked by #197

Metadata: Scan only what's needed #201

tasket opened this issue May 30, 2024 · 0 comments
Labels
enhancement New feature or request
Milestone

Comments

@tasket
Copy link
Owner

tasket commented May 30, 2024

Problem

Wyng currently scans all archive metadata items (root, volumes and sessions) on startup. This impacts startup times and is also sub-optimal when encountering or recovering from errors.

Solution

Map the commands to metadata requirements, such as receive only needing one volume's metadata at a time. When receiving older sessions, it may not be necessary to instantiate all session objects / infos. verify, diff and list could have the same/similar metadata usage profile, while delete would be a special case where as little as zero volinfos could be used if the user is able to specify a volume id. And monitor would be another special case, since it is a goal to be able to run it without authenticating.

The profile for send would not be as flexible as the others, since the send workflow may call on both deduplication (which requires every bit of metadata) and pruning operations.

The arch-check command could have a similar profile to verify depending on the parameters used.

Notes

A generic load() method for objects may provide needed flexibility.

Its probable this issue will focus mainly on session objects, although being flexible with volume object should also be a goal.

@tasket tasket added the enhancement New feature or request label May 30, 2024
@tasket tasket added this to the v0.9 milestone May 30, 2024
tasket added a commit that referenced this issue Jun 7, 2024
tasket added a commit that referenced this issue Jun 8, 2024
tasket added a commit that referenced this issue Jun 8, 2024
tasket added a commit that referenced this issue Jun 8, 2024
tasket added a commit that referenced this issue Jun 9, 2024
Handle corner case if no sessions / no good sessions

Update storage vol list if empty

Retain sync proc reference
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant