Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finalize "Pending" zarr if all upload URLs must have timed out by now #2051

Open
yarikoptic opened this issue Oct 17, 2024 · 0 comments
Open
Labels
zarr Issues with Zarr hosting/processing/etc.

Comments

@yarikoptic
Copy link
Member

Instigator:

which summarizes as - we do not adjust "modified" time stamp of the dandiset even if zarr is modified (some files uploaded/changed) whenever it is never finalized. I think we should have some periodic job which would go through Pending zarrs, and if it was put into Pending state longer before the time window we allow for uploads/changes to happen in zarr -- "forcefully" finalize it in current state

NB in principle, whenever we have versioning implemented, we could revert back by removing versions of modifications to the keys and DeleteMarkers possibly added. But since we do not track to that level ATM, we cannot revert AFAIK. Also reversal might introduce the need to add similar reversal in backups2datalad, thus the easiest is just to finalize.

I think it might relate to

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
zarr Issues with Zarr hosting/processing/etc.
Projects
None yet
Development

No branches or pull requests

1 participant