Repository navigation
fix(worker): surface the blocked elastic drive when the world is unprovable - #739
Merged
Merged
Conversation
…ovable With no operation record, an elastic pass that cannot prove the live world finished silently: the observation kept the proof failure and the ElasticResize condition leveled to True/NoRefusal, so the deployment read healthy while the engine and the pod set held a width the spec no longer asks for. No condition changed, no record was created, and no later pass had any more drive than the one that parked, so the wedge held indefinitely. The unprovable park is now loud when it matters. With no record and a live width that disagrees with the spec, or cannot be named at all, the pass writes the condition Unknown with a new WorldUnprovable reason whose message carries the desired width, the observed width and the proof that failed. Unknown because nothing was refused, and a distinct reason because True/NoRefusal is the healthy state that kept the park invisible. A live width that agrees with the spec owes no corrective scale and parks as quietly as before, and the observation reason keeps the proof's own failure either way. The blocked state acts on nothing - no record, no engine command, no member touched - because acting on unproven ranks is the safety design. Recovery is ordinary: a pass that proves the world reaches the corrective block, recreates the resize, and levels the condition back through the record-driven writer. No timers and no requeue changes; the ordinary observation cadence re-drives the pass. Fixes #738 Signed-off-by: thxCode <thxcode0824@gmail.com>
|
🔍 OpenCodeReview found 1 issue(s) in this PR.
📄
|
Collaborator
Author
|
Review round summary (all three feedback buckets read to exhaustion: 0 top-level reviews, 0 inline threads, 1 issue comment — the OpenCodeReview summary). Triaged items: 1.
Fixed: 0. Not fixed (already satisfied): 1. Deferred: 0. |
thxCode
added a commit
that referenced
this pull request
Oct 7, 2026
…apes (#740) The ElasticResize condition and the behaviors that merged in #727, #732, #737 and #739 had no documentation. Add the condition to the status vocabulary and describe the refusal-retry bound, the two self-healing stale shapes, and the diagnosis entry in Elastic EP. Signed-off-by: thxCode <thxcode0824@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind bug
/area worker
What this PR does / why we need it:
With no durable operation record, an elastic pass that cannot prove the live world finished silently: the observation kept the proof failure and the
ElasticResizecondition leveled toTrue/NoRefusal, so the deployment read healthy while the engine and the pod set held a width the spec no longer asks for. No condition changed, no record was created, and no later pass had any more drive than the one that parked, so the wedge held indefinitely — observed live with desired=2 against a live width of 4 while the effective layer reported "native rank identities are unknown".The unprovable park is now loud when it matters. With no record and a live width that disagrees with the spec, or cannot be named at all, the pass writes the
ElasticResizecondition Unknown with a new WorldUnprovable reason whose message carries the desired width, the observed width and the proof that failed. Unknown because nothing was refused, and a distinct reason becauseTrue/NoRefusalis the healthy state that kept the park invisible. A live width that agrees with the spec owes no corrective scale and parks as quietly as before, and the observation reason keeps the proof's own failure either way.The blocked state acts on nothing — no record, no engine command, no member touched — because acting on unproven ranks is the safety design. Recovery is ordinary: a pass that proves the world reaches the corrective block, recreates the resize, and levels the condition back through the record-driven writer. No timers and no requeue changes; the ordinary observation cadence re-drives the pass. Both directions are pinned by tests: the blocked surfacing for an unknown live width and for a wider live width that fails its forward proof, and the level-back once the world proves again.
Which issue(s) this PR links to:
Fixes #738
Special notes for your reviewer:
Unknown/WorldUnprovable) besideFalse/ScaleRefusedandTrue/NoRefusal; it is declared and documented beside the convergence that decides it, and nothing outside the elastic reconcile reads the condition, so no consumer changes.#734); the re-probe itself is untouched, and a narrower or unknown live world keeps its hold — it now just says so on the condition instead of reading healthy.Does this PR introduce a user-facing change?