ci(validate-oc-ampere): add e2e cluster validation workflow

[dark-vex]

Summary

• Adds validate-oc-ampere.yml, giving oc-ampere its first live e2e coverage (real k3s + real flux install + real reconcile-to-Ready), matching the mechanism already merged for kubenuc/k8s-vms-daniele (ci(kubenuc,k8s-vms): derive top-level cluster resources instead of hand-enumerating #1790-ci(validate-flux-render): work around flate's concurrent-dispatch livelock #1795) and companion PR ci(validate-k3s-rabbit): add e2e cluster validation workflow #1796 for k3s-rabbit.
• Reuses Mechanism 1 (top-level-resource loop) verbatim. Extends Mechanism 2 (fail-closed per-app loop) with a new fourth resolution tier: a dirname()-based fallback that parses the root apps/kustomization.yaml for apps with no per-app deploy.yaml of their own — needed here for ngx-webhook.
• 2 of 4 real apps deploy for real: flux-operator (real-path fallback) and ngx-webhook (new tier-4 resolver, no credentials/outbound calls, NodePort-only). system-upgrade-controller and teleport-agent are excluded for the same live-external-side-effect reasons as k3s-rabbit's.
• flux-operator pre-FluxInstance risk, resolved: rendered the pinned chart (helm template ... --version 0.57.0 --values '{}') — it installs only a Deployment/RBAC/CRDs/NetworkPolicy/Service, no Helm hooks. The Deployment's only arg is --log-level=info; RBAC includes a broad get/list/watch rule (used for its FluxReport cross-cluster status aggregation) and a patch/reconcile rule scoped to kustomize.toolkit.fluxcd.io/helm.toolkit.fluxcd.io (used by its optional Web UI/MCP-server manual-reconcile-trigger feature — nothing in this workflow calls those endpoints). Nothing fires automatically at pod startup absent a FluxInstance CR. flux-instance.yaml itself stays skip-listed in this job (which bootstraps Flux via plain flux install instead) — documented in-workflow why that must never change.
• States the ARM64 gap plainly: production oc-ampere's compute is genuinely ARM64 (terraform/oci/k8s-armchair, VM.Standard.A1.Flex); this run validates manifest/reconciliation correctness on the (undocumented, presumed x86_64) self-hosted runner, not ARM64 image availability. Named images: nginxinc/nginx-unprivileged:1.31-alpine, ghcr.io/controlplaneio-fluxcd/flux-operator:v0.57.0.
• Comment-only touch to clusters/oc-ampere/apps/ngx-webhook/manifests/deploy.yml exercises the tier-4 resolver on this workflow's first real run.

Test plan

• Mechanism 1 parser tested locally against the real clusters/oc-ampere/kustomization.yaml (resolves to system-upgrade-controller.yaml only; flux-instance.yaml/cluster-vars.yaml correctly skipped)
• Tier-4 resolver tested locally against the real apps/kustomization.yaml with APP=ngx-webhook (resolves via dirname() to ngx-webhook/manifests)
• Synthetic multi-entry ambiguity case tested (correctly identifies >1 match, would fail closed)
• EXCLUDED_APPS membership verified for both excluded apps; flux-operator/ngx-webhook confirmed not excluded
• flux-operator chart rendered locally, no hooks, idle-until-FluxInstance confirmed (see summary)
• YAML + pre-commit hooks pass
• Real CI run on this PR: confirm all 21 chart sources reconcile within the charts Kustomization's wait timeout (first time this repo exercises that volume), flux-operator and ngx-webhook both reach Ready, system-upgrade-controller/teleport-agent correctly skipped

🤖 Generated with Claude Code

[dark-vex]

Independent Codex review

Ran an independent Codex review (via the local Codex CLI, not the cloud /code-review ultra) against this PR before merge, primed with the plan's flux-operator and tier-4-resolver claims to verify/refute rather than re-derive.

Real bug found and fixed (66d5dfd4): DEPLOY_APPS is derived purely from git diff against main, and this PR's diff only touched ngx-webhook/'s directory — so despite the workflow correctly listing flux-operator as a non-excluded, real-path-deployed app, this PR's own CI run would never have actually exercised that deploy. The workflow's design was correct (a future PR touching apps/flux-operator/ would trigger it), but this specific verification run would have proven nothing about it — the exact class of gap flagged in the plan re: PR #1791's lesson, just missed for the second app. Fixed by adding the same kind of comment-only touch already used for ngx-webhook, this time under apps/flux-operator/. Confirmed locally against origin/main...HEAD that DEPLOY_APPS now resolves to both flux-operator ngx-webhook.

Minor fix: tier-4 resolver now also checks the matched apps/kustomization.yaml entry resolves to a real file (-f check) before computing dirname(), not just that it contains a / — closes an edge case where a future directory-style entry with a / in its name could've been misclassified as a file path instead of failing at the resolver boundary.

Confirmed correct, no action needed:

• Tier-4 resolver's fail-closed paths (zero-match, multi-match, bare-directory) all verified correct against the real manifest.
• EXCLUDED_APPS reasons for system-upgrade-controller/teleport-agent verified against the real manifests (cordon: true, real Teleport proxyAddr/production role config).
• flux-instance.yaml/cluster-vars.yaml skip-listing confirmed safe.

Not independently re-verified by Codex (its sandbox couldn't reach ghcr.io to run helm template itself): the specific claim that the flux-operator chart renders no Helm hooks and stays idle absent a FluxInstance CR. That claim stands on my own local helm template run (documented in the PR description above), not on Codex's own reproduction — flagging so this isn't mistaken for independently double-verified.

Noted but not changed (pre-existing convention shared with validate-kubenuc.yml/validate-k8s-vms.yml, not a regression introduced here, out of scope for this PR): unpinned curl | sh k3s installer, GITHUB_TOKEN passed as a CLI arg (visible via process listing on the shared self-hosted runner), and the hardcoded Harbor mirror endpoint. Worth a follow-up hardening pass across all four e2e workflows if desired, but changing it unilaterally here would diverge from the established pattern without a broader decision.