fix: reorder Kata CI gate before full suite + fix 4 flaky sandbox tests - #905
Merged
Conversation
sabbour
force-pushed
the
fix/ci-kata-gate-and-flaky-tests
branch
from
August 25, 2026 12:45
856f35a to
a026828
Compare
sabbour
changed the base branch from
dev
to
perf/ci-caching-and-job-consolidation
August 25, 2026 12:48
sabbour
force-pushed
the
fix/ci-kata-gate-and-flaky-tests
branch
from
August 25, 2026 14:34
dbedf39 to
c8cd89b
Compare
sabbour
force-pushed
the
fix/ci-kata-gate-and-flaky-tests
branch
from
August 25, 2026 14:48
c8cd89b to
f05bb02
Compare
sabbour
force-pushed
the
fix/ci-kata-gate-and-flaky-tests
branch
from
August 25, 2026 15:30
f05bb02 to
ddf6c67
Compare
sabbour
force-pushed
the
fix/ci-kata-gate-and-flaky-tests
branch
from
August 25, 2026 15:40
ddf6c67 to
74519c4
Compare
… hard ResolveSandboxProcessAsync threw immediately when the bwrap child was observed sharing the executor's process group. bwrap --new-session calls setsid() in the child after fork, creating a short window where the child still has the parent PGID. Convert the hard throw to a continue so the existing 10-second timeout covers the genuine misconfiguration case. Fixes intermittent KataPreviewStop_SignalsActualSandboxProcessGroupWithTerm. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
Poll for the TERM-handler's marker file content instead of reading it once immediately after StopPreviewProcessAsync returns: the sidecar Ack only proves it signalled the sandboxed process group, not that the trap's own file write has landed on disk yet. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
StartListener queued its accept-and-reply on Task.Run, which competes for the shared ThreadPool with every other parallel test. A starved pool could delay Accept() past the probe's 750ms receive timeout and make a real builder look unreachable. Run it on a dedicated background Thread instead, and block until that thread has actually started before returning, removing pool-scheduling latency from the race. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
The bwrap mount plan for the read-back exec resolves and binds the just-created worktree admin dir fresh, so there is a narrow window where a healthy sandbox can observe it before it has fully settled. Split the mutating append (run once) from the read (git status/diff, idempotent) and poll the read up to 3 attempts with a short delay instead of asserting once on a result entangled with that timing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
A single ReadLineAsync raced the workload's own stdout flush: shell/ setsid startup can interleave a blank line or chatter ahead of the echo depending on scheduling, so asserting the very first line read is the expected one was flaky. Poll a bounded number of lines within the existing 30s budget for the expected line instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
…ace after setsid poll Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
…ndbox fix bwrap /bin/bash intermittently not visible in CI sandbox environment. Tracked separately; unrelated to setsid race fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 11857092-1327-41bd-8708-b2ca85e674dd
sabbour
force-pushed
the
fix/ci-kata-gate-and-flaky-tests
branch
from
August 25, 2026 15:54
74519c4 to
dd90e92
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The .NET tests job runs the full ~3,500-test suite (~6.9 min avg) BEFORE the ~40-test Kata runtime gate (~11s). When the gate fails, ~6.9 minutes of otherwise-passing work is discarded.
Changes
Production fix
CI structural fix
ick-fields/retry@v3\ (\max_attempts: 2) to absorb runner-level flakiness (apt-get network hiccups, transient namespace setup) without masking real test failures. The full suite step is NOT retried.
ode scripts/ci/validate.mjs --profile ci --area dotnet\ into explicit restore/build/test steps (documented in CONTRIBUTING.md as the equivalent locked-restore/one-build/test-exact-outputs trio) so the Kata gate can reuse the same build output ahead of the full suite.
Flaky test fixes
Changesets
Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com