Add a CircleCI job that builds images and runs the application tests against Kubernetes

[fniessink]

Prerequisite

This plan assumes ICTU/quality-time#13531 ("Build (not download) images in the Kubernetes application-tests workflow") is already done (which itself required #13530). So when this work starts the repo already has:

• .github/workflows/application-tests-k8s.yml — builds branch images (:ci), loads them into minikube, deploys the chart with CI overrides, port-forwards, and runs the application tests against Kubernetes.
• helm/values-application-tests.yaml — CI values: image.tag: ci + image.pullPolicy: Never for all components, plus api_server.env.LDAP_URL: ldap://quality-time-ldap:389.
• helm/deploy-ci.yaml — CI-only test-LDAP (quality-time-ldap) and Selenium (selenium) Deployments/Services (and, pre-this-plan, the db + ldap Secrets).
• Configurable imagePullPolicy in the chart; the testldap ldap compose service named ictu/quality-time_testldap.
• Application tests parametrized via WWW_URL / WWW_BROWSER_URL / SELENIUM_URL.

Context

The repo runs CI on both GitHub Actions and CircleCI. CircleCI (.circleci/config.yml) already has an application_tests job that mirrors the Docker GitHub workflow: a machine executor that does docker compose build && docker compose up -d and runs the tests in a uv container joined to the proxy's network. There is no CircleCI job that exercises the Kubernetes deployment.

This plan adds that: a CircleCI job that builds the branch images and runs the application tests against a minikube Kubernetes deployment — the CircleCI counterpart of application-tests-k8s.yml.

Per the chosen approach, the build→deploy→test logic is extracted into one portable shell script that both CI systems call, so the two never drift. Each CI keeps only what is genuinely CI-specific: provisioning the cluster and toolchain.

---

Changes

1. New shared script ci/application-tests-k8s.sh

A portable bash script (long-form options, per project convention) that assumes a reachable Kubernetes context (kubectl/helm/docker on PATH, minikube started) and does:

1. Create CI secrets imperatively (portable, no GitHub-only action, no yq):

   ldap_password=$(openssl rand --hex 16)
   kubectl create secret generic quality-time-db-secret \
     --from-literal=DATABASE_USERNAME="$(openssl rand --hex 8)" \
     --from-literal=DATABASE_PASSWORD="$(openssl rand --hex 16)"
   kubectl create secret generic quality-time-ldap-secret \
     --from-literal=LDAP_LOOKUP_USER_DN=cn=admin,dc=example,dc=org \
     --from-literal=LDAP_LOOKUP_USER_PASSWORD="$ldap_password"
   

   Random hex values avoid the MongoDB/SASLprep control-character problem that Run the application tests against Kubernetes as well as Docker #13530 used printf | base64 to dodge, and kubectl base64-encodes automatically. testldap's cn=admin password comes from the same quality-time-ldap-secret (wired in Run the application tests against Kubernetes as well as Docker #13530's deploy-ci.yaml), so lookup-user and admin passwords stay identical.
2. Build images (tag ci):
   QUALITY_TIME_VERSION=ci docker compose --progress quiet --file docker/docker-compose.yml --file docker/docker-compose.ci.yml --project-name quality-time build
3. Load into minikube (containerd): loop minikube image load <image> over the 8 ictu/quality-time_*:ci images (7 components + ictu/quality-time_testldap:ci).
4. Deploy: kubectl apply --filename helm/deploy-ci.yaml; helm dependency build helm; helm upgrade --install --render-subchart-notes quality-time helm --values helm/values-application-tests.yaml.
5. Wait: kubectl wait for all pods Ready and all deployments Available (covers quality-time-ldap and selenium).
6. Port-forward service/quality-time-www 8080:8080 and service/selenium 4444:4444 in the background, then poll until both ports accept connections.
7. Run the tests:
   docker run --network=host --env WWW_URL=http://localhost:8080 --env SELENIUM_URL=http://localhost:4444 --env WWW_BROWSER_URL=http://quality-time-www:8080 ... ghcr.io/astral-sh/uv:python3.14-bookworm tests/application_tests/test.sh

Failure diagnostics (kubectl get pods/events, describe, per-pod logs) go in a small companion ci/k8s-diagnostics.sh, called by each CI on failure.

2. Adjust the #13531 artifacts to use the shared script

• .github/workflows/application-tests-k8s.yml: replace the inline build/load/deploy steps (and the VALUE_CHANGES + fjogeleit/yaml-update-action secret step) with bash ci/application-tests-k8s.sh. Keep: checkout, medyagh/setup-minikube, artifact upload, and if: failure() → bash ci/k8s-diagnostics.sh. (kubectl/helm are preinstalled on GitHub-hosted Ubuntu runners; minikube via the action.)
• helm/deploy-ci.yaml: remove the two Secret documents (now created imperatively by the script); keep the LDAP + Selenium Deployments/Services. Both CIs create the secrets the same way.

Ordering note: since #13531 is not merged yet, these two edits are cleanest folded into #13531 itself. If #13531 lands first, this plan adjusts them.

3. New CircleCI job application_tests_k8s (.circleci/config.yml)

• machine: image: default, resource_class: large (bump to xlarge if minikube + 8 images + Selenium's 2 GB shm are resource-starved).
• Steps: checkout; docker login with the QualityTime context creds ($DOCKERHUB_USERNAME/$DOCKERHUB_PASSWORD) to avoid pull-rate limits; install pinned kubectl, helm, minikube (curl, SHA/version-pinned per repo convention); minikube start --driver=docker --container-runtime=containerd (matching the GitHub setup); bash ci/application-tests-k8s.sh; store_artifacts for build/; and a when: on_fail step running bash ci/k8s-diagnostics.sh.

4. Rename the CircleCI Docker job for symmetry

• Rename application_tests → application_tests_docker in .circleci/config.yml and update its reference in the build_test workflow.
• Add application_tests_k8s to the build_test workflow with context: QualityTime (alongside application_tests_docker, feature_tests, unittests).

5. Docs

Update docs/src/development.md (the "Application tests" section, ~line 174) to note the application tests run against Kubernetes on both GitHub Actions and CircleCI, via the shared ci/application-tests-k8s.sh.

---

Critical files

• new ci/application-tests-k8s.sh — shared build→secrets→load→deploy→port-forward→test logic.
• new ci/k8s-diagnostics.sh — shared on-failure dump.
• .circleci/config.yml — new application_tests_k8s job + rename application_tests → application_tests_docker + workflow wiring.
• .github/workflows/application-tests-k8s.yml — call the shared script instead of inline steps.
• helm/deploy-ci.yaml — drop the Secret documents (created imperatively now).
• docs/src/development.md — document the dual-CI k8s run.

Reuse (don't re-create)

• helm/values-application-tests.yaml, the LDAP/Selenium manifests in helm/deploy-ci.yaml, the :ci image tags, and the parametrized tests — all from Run the application tests against Kubernetes as well as Docker #13530/Build (not download) images in the Kubernetes application-tests workflow #13531; the script just orchestrates them.
• The compose build pair (docker/docker-compose.yml + docker/docker-compose.ci.yml) — same invocation the existing CircleCI application_tests job already uses, just with QUALITY_TIME_VERSION=ci.

Trade-offs / notes

• Edits Build (not download) images in the Kubernetes application-tests workflow #13531's artifacts: moving secret creation into the shared script removes GitHub's yaml-update-action step and the Secret stanzas from helm/deploy-ci.yaml. This simplifies both CIs but means this plan touches files Build (not download) images in the Kubernetes application-tests workflow #13531 introduces — best coordinated by folding into Build (not download) images in the Kubernetes application-tests workflow #13531 if not yet merged.
• CI cost: CircleCI now builds all 8 images and runs minikube on a machine executor — heavier and slower than the lightweight docker-executor jobs; large/xlarge resource class required.
• minikube provisioning stays per-CI: GitHub uses the setup-minikube action; CircleCI installs minikube manually. Only this thin provisioning layer differs; everything downstream is shared.
• Secret approach change: imperative kubectl create secret with random hex replaces base64-of-$RANDOM; hex is control-character-free, preserving the SASLprep safety Run the application tests against Kubernetes as well as Docker #13530 cared about.

Verification

1. Script lint: bash -n ci/application-tests-k8s.sh and shellcheck ci/application-tests-k8s.sh ci/k8s-diagnostics.sh pass.
2. CircleCI config valid: circleci config validate accepts .circleci/config.yml; the build_test workflow lists application_tests_docker, application_tests_k8s, feature_tests, unittests.
3. Local end-to-end (containerd minikube): minikube start --driver=docker --container-runtime=containerd; bash ci/application-tests-k8s.sh → secrets created, 8 :ci images loaded (minikube image ls | grep :ci), all pods Ready, tests pass, build/ artifacts written.
4. Parity: the GitHub application-tests-k8s run (now calling the same script) and the CircleCI application_tests_k8s run produce the same test outcome on the same commit.
5. Branch sensitivity: a trivial visible change in a component appears in both CIs' k8s runs, confirming branch code (not published images) is under test.
6. Per the arm64/amd64 memory note: both CI runners are amd64; trust them for runtime-constraint behavior in images embedding browsers/native binaries (renderer, selenium).