-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle transmission mode reference #419
Conversation
…ust be provided at deploy time
…e outputs or multi-by-ref (relates to #376)
…download option of those file links
if job.status in JOB_STATUS_CATEGORIES[StatusCategory.RUNNING]: | ||
# signal to stop celery task. Up to it to terminate remote if any. | ||
LOGGER.debug("Job [%s] dismiss operation: Canceling task [%s]", job.id, job.task_id) | ||
celery_app.control.revoke(job.task_id, terminate=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that the revoke command is asynchronous, could the cleanup following it (removing logs, xml, dir) be done prematurily while the process is still spitting logs waiting to be killed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is a possibility. I haven't explored this much.
If we encounter this, we could think of a way to handle it. Probably need to run another task with a delay to do a cleanup with below checks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In fact, the succeeded jobs also leave their stuff, so a mechanism to clean that should already be in place anyway. For the typical birds dumping the result in the output directory, there was a cron job for that. Does it apply to Weaver also?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The results that were generated by pywps and cwltool should be all cleaned up following execution by weaver after it moved them to wps-outputs. There should be 3 items {jobId}.log
, {jobId}.xml
(WPS status), {jobId}/
(output results).
The dismiss cleanup applied here is more specific for OGC Dismiss operation that requests explicitly a cleanup of those outputs, given a user downloaded them.
If the cron job runs inside wps-outputs/
for each birds, then it should pick up weaver as well, but there is not a cron specifically within weaver.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Goog PR. There is still an open comment, but it's not a deal breaker.
Changes
Many adjustments for OGC compliance.
results
directly when running insync
mode (instead ofstatus
info)Link
header for outputs requested withtransmissionMode=reference
References
response: raw
execution results #376(raw single
value
output and/or manyreference
working, multipart not yet added)