Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make 'mop' suitable for cron job #130

Open
gsaksena opened this issue Aug 7, 2019 · 0 comments
Open

Make 'mop' suitable for cron job #130

gsaksena opened this issue Aug 7, 2019 · 0 comments

Comments

@gsaksena
Copy link

gsaksena commented Aug 7, 2019

Running mop may disrupt workflows that are currently running, so today one needs to ensure everything is quiet before running it.

While this is not an issue for workspaces you want to mothball, it means you cannot run it asynchronously as part of a cron job (or Google Cloud Scheduler job).

One way to do that might be to only delete objects created older than a certain age. Eg as tasks are limited to 7 days, and people tend to not like really long workflows, limiting the purge to objects created > 10 days ago will likely avoid clobbering a running pipeline. If the user knows the pipelines are all much shorter, a shorter timeframe could be selected. A longer timeframe, eg 60 days, might be chosen if they want to ensure call caching on their current project would not be impacted.

Tighter coupling between mop and the Cromwell/Terra databases might offer a cleaner solution, but if that's too complicated the easier solution above would still be an improvement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant