ci(validate-k3s-rabbit): add e2e cluster validation workflow - #1796
Merged
Conversation
k3s-rabbit had zero live e2e coverage (only offline flate/kubeconform schema checks) despite being a real production Flux Operator cluster, unlike kubenuc and k8s-vms-daniele which already got real k3s+Flux e2e runs. Ports the same top-level-resource loop and fail-closed per-app loop, but all 3 real apps (fluxcd, system-upgrade-controller, teleport-agent) are excluded — each has a live external side-effect (Slack/GitHub notifications, a real k3s upgrade-channel call with node-cordon capability, a real Teleport tunnel join) — so this run proves Flux bootstrap, GitRepository resolution, the chart source, and the top-level system-upgrade-controller install reaching Ready, plus fail-closed coverage the moment a 4th app is ever added. The comment-only touch to apps/fluxcd/notifications.yaml exercises the CHANGED-parsing fix and the EXCLUDED_APPS skip path on this workflow's first real CI run. Signed-off-by: Daniele De Lorenzi <2905124+dark-vex@users.noreply.github.com>
This was referenced Aug 6, 2026
The real CI run on this PR failed: "Check resource deployment status"
ran `flux get helmreleases -A` with no `|| true`, and flux's CLI exits
non-zero when it finds zero matching objects ("no HelmRelease objects
found in any namespace") — under GitHub Actions' default bash
`set -eo pipefail`, that failed the whole step and the job.
On k3s-rabbit specifically this isn't a transient flake — zero
HelmReleases exist by design on every run (3/3 real apps excluded, see
the "Deploy infra baseline and changed apps" step's own header
comment, which already claimed this step doesn't gate the job's exit
code; it does, until now). Add `|| true` to both `flux get` calls,
consistent with the "Check for failed reconciliations" step just below
it, which already treats itself as informational-only the same way.
Signed-off-by: Daniele De Lorenzi <2905124+dark-vex@users.noreply.github.com>
dark-vex
enabled auto-merge
August 7, 2026 08:11
4 tasks
dark-vex
added a commit
that referenced
this pull request
Aug 7, 2026
ci(validate-k3s-rabbit): add e2e cluster validation workflow
dark-vex
added a commit
that referenced
this pull request
Aug 7, 2026
flux-operator landed on k3s-rabbit via separate PRs (#1799, #1804, #1805) while the e2e workflow PR (#1796) was still rebasing on top of them, so it never appeared in that PR's own diff and no live CI run has exercised the tier-3 real-deploy path for it here — unlike oc-ampere, where an identical comment-only touch (PR #1797, commit 66d5dfd) already forced that coverage. A comment-only touch on flux-operator's HelmRelease forces it into this PR's own DEPLOY_APPS so this run finally exercises that path. Also corrects several comments/docs that assumed k3s-rabbit deploys zero real apps by design, stale since flux-operator's addition. Safety re-verified independently this session via a fresh live `helm template` render of chart v0.57.0: zero Helm hooks, no FluxInstance object; the Deployment just starts the operator controller, which idles until a FluxInstance CR exists. k3s-rabbit's root kustomization.yaml has no flux-instance.yaml entry, so that CR is never created during an e2e run — same as oc-ampere. Signed-off-by: Daniele De Lorenzi <2905124+dark-vex@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
validate-k3s-rabbit.yml, giving k3s-rabbit its first live e2e coverage (real k3s + realflux install+ real reconcile-to-Ready), matching the mechanism already merged for kubenuc/k8s-vms-daniele (ci(kubenuc,k8s-vms): derive top-level cluster resources instead of hand-enumerating #1790-ci(validate-flux-render): work around flate's concurrent-dispatch livelock #1795).fluxcd,system-upgrade-controller,teleport-agent) are excluded — each has a real live external side-effect (Slack/GitHub notifications, a k3s upgrade-channel call with node-cordon capability, a Teleport tunnel join). The app loop deploys zero apps by design on every run today; see the in-workflow header comment for exactly what the run still proves and what conditions actually gate the job's exit code.CHANGED-parsing fix (require a further/afterapps/) from day one — this bit the kubenuc/k8s-vms-daniele workflows twice before it became structural there.clusters/k3s-rabbit/apps/fluxcd/notifications.yamlexercises theEXCLUDED_APPSskip path and theCHANGED-parsing fix on this workflow's first real run — a diff that only touched the workflow file would prove nothing about the per-app loop (same lesson as PR ci(kubenuc): fail-closed per-app e2e loop with documented EXCLUDED_APPS #1791).Test plan
clusters/k3s-rabbit/kustomization.yaml(resolves tosystem-upgrade-controller.yamlonly,flux-instance.yaml/cluster-vars.yamlcorrectly absent/skipped)CHANGEDgrep tested against a synthetic diff touching onlyapps/kustomization.yaml(yields zero bogus app names) and against a diff touching real app files (yields correct app names)EXCLUDED_APPSmembership verified for all 3 real appscheck-yaml,trailing-whitespace,gitleaks) passcharts(teleport-charts) reconciling, top-levelsystem-upgrade-controllerKustomization reaching Ready, and theEXCLUDED_APPSskip path firing for all 3 apps🤖 Generated with Claude Code