Workflow hit runahead on reload but can't figure out why #6517
Labels
bug
Something is wrong :(
needs reproducing
A bug report that does not yet have a reproducible example
Milestone
Description
Sorry for the vaguenss, I'm trying to figure out more information, but am juggling a lot. This is for multiple workflows.
They were running with CYLC_VERSION=8.3.5
I stopped them
I did a
cylc vr
with CYLC_VERSION=8.3.6 (and various small updates)Log files are only showing WARNING and ERROR, no INFO (frustratingly I had the
-q
option on), so there isn't much information in thereWorkflow has stalled, hitting a runahead limit - but I can't tell how or why or what runahead limit is hit because there is nothing in the logs about it. I don't think the workflow recognises it is stalled. The task in the image below which has run I manually triggered to see it would register.
In the above image, the
start_ops
task does have an xtrigger for it, no other pre-reqs. The xtrigger was not satisfied, but could have been if it actually run (I believe xtriggers don't run if the runahead is met?)If I make a job fail, I see the WARNING message printed, so logs are still updating
3 cycles ran without issue, but the fourth has not started
There is no graph offset that goes more than one cycle (6 hourly cycling model)
Reproducible Example
Haven't got one yet.
Expected Behaviour
Suite wouldn't stall, or if it does, at least say it did in the logs. Given it hasn't run for 12 hours, it has stalled.
The text was updated successfully, but these errors were encountered: