Skip to content

testing.md

Latest commit

 

History

History
155 lines (107 loc) · 8.37 KB

testing.md

File metadata and controls

155 lines (107 loc) · 8.37 KB
Raw

Testing

The Perses Operator uses unit, integration, end-to-end (e2e), and alerting rule tests.

Some test commands use tools installed locally under bin/. Run make build-tools to install all tools, or let the Makefile targets (e.g. make test-unit, make test-integration) install them automatically.

Unit Tests

Unit tests validate individual functions and utilities using standard Go testing with testify or Ginkgo.

# Run all unit tests
make test-unit

# Run a single test by name (testify)
go test ./internal/metrics/... -run TestNewMetrics -v

# Run a single test by name (Ginkgo)
# --focus matches against Describe/It block descriptions
bin/ginkgo --focus "GetVolumes" -v ./internal/perses/common/...

Integration Tests

Integration tests use Ginkgo with envtest to spin up a lightweight API server with CRDs installed. These tests validate controller reconcile logic, CRD validation, and resource management.

# Run all integration tests
make test-integration

# Run a single integration test by name
# --focus matches against Ginkgo Describe/It block descriptions
KUBEBUILDER_ASSETS="$(bin/setup-envtest use -p path)" \
  bin/ginkgo --focus "should successfully reconcile" -v ./controllers/...

E2E Tests (kuttl)

E2E tests use kuttl to validate the operator end-to-end in a real kind cluster, including deployment, resource synchronization, and cleanup.

Prerequisites

  • Go
  • kind — use the version pinned by kind-version in .github/env to match CI and the configured Kubernetes node image
  • Docker or Podman — the Makefile auto-detects the available container runtime
  • kubectl

Before setup, compare the expected versions with your installed kind:

make e2e-versions
kind version

Install or upgrade kind to the pinned version if they differ. The Makefile uses kind from your PATH; neither make build-tools nor setting KIND_VERSION installs or upgrades it. Older kind versions may generate kubeadm configuration APIs that the configured node image no longer supports.

Quick Start

make e2e-setup    # Create cluster + deploy operator
make test-e2e     # Run tests
make e2e-cleanup  # Tear down

make e2e-setup reuses a cluster with the configured name if one already exists. Cluster-listing and creation failures stop setup and show the underlying error instead of continuing to deployment.

When iterating:

  1. Run make e2e-setup once to create the cluster and deploy the operator.
  2. Run make test-e2e to run the tests.
  3. After code changes, run make e2e-deploy to rebuild and redeploy the operator, then make test-e2e again.

Configuration

The following Makefile variables can be overridden to customize the e2e environment:

Variable Default Description
KIND_CLUSTER_NAME kuttl-e2e Name of the kind cluster
E2E_TAG Git short SHA Image tag for the operator
E2E_IMG docker.io/persesdev/perses-operator:<E2E_TAG> Full operator image reference

make e2e-deploy applies the config/e2e kustomize overlay (on top of config/default), which sets --resource-sync-interval=15s so drift-healing tests (see dashboard-api-drift) finish quickly. Production installs keep using config/default (5 minute default).

Test Structure

Tests are under test/e2e/ following the kuttl convention. Kuttl creates a random namespace per test case for isolation. Steps run sequentially.

test/e2e/
├── kuttl-test.yaml
├── dashboard-api-drift/    # Recreate dashboard deleted from Perses API (periodic sync)
├── dashboard-tags/         # Dashboard tag annotation sync
├── encryption-key-replay/  # Credential recovery after key changes and API-secret deletion
├── global-datasource/      # Global datasource sync
├── multi-instance-sync/    # Dashboard/datasource sync across instances
├── namespace-isolation/    # Namespace-scoped resource isolation
├── perses-deployment/      # database.sql → Deployment
├── perses-emptydir/        # emptyDir storage
├── perses-oauth-tls/       # Native OAuth with distinct API/token TLS hostnames
├── perses-statefulset/     # database.file → StatefulSet with PVC
├── perses-volumes/         # Custom volumes and volume mounts
└── resource-update/        # Update existing resources and assert reconciliation
  • To add a new test case, create a directory under test/e2e/.
  • To add steps to an existing test case, add numbered files (e.g. 05-install.yaml, 05-assert.yaml).

OAuth/TLS transport regression

perses-oauth-tls runs HTTPS Perses with file storage and native OAuth authentication. A local nginx fixture supplies upstream OAuth responses and proxies Perses's own token endpoint through a second HTTPS hostname. Cert-manager issues separate API and token-proxy certificates with disjoint DNS SANs, signed by the same test CA. Certificate verification remains enabled for the operator, proxy, and verification Job.

The case verifies dashboard reconciliation and authenticated API retrieval, and rejects unauthenticated requests and the fake upstream token. File storage exercises pod-direct API connections: the API transport must verify the Perses Service hostname, while OAuth token acquisition must verify the token URL's hostname independently. Reusing the API transport for OAuth requests fails this case with a certificate hostname mismatch.

After make e2e-setup, run only this case with:

bin/kubectl-kuttl test --config test/e2e/kuttl-test.yaml --test perses-oauth-tls

The fixture is namespace-isolated and requires no external identity provider or ingress controller. It pulls nginx:1.28-alpine and python:3.13-alpine in addition to the Perses image.

Debugging

If setup fails while creating the cluster, check the kind error output and compare kind version with make e2e-versions. An error such as your configuration file uses an old API spec: "kubeadm.k8s.io/v1beta3" indicates that kind generated a kubeadm configuration unsupported by the node image; install the kind version pinned in .github/env and retry setup.

# Operator logs
kubectl logs -n perses-operator-system deployment/perses-operator-controller-manager -f

# Run a single test case
bin/kubectl-kuttl test --config test/e2e/kuttl-test.yaml --test perses-statefulset

Alerting Rule Tests

Note

Requires promtool.

Alerting rules are validated using promtool. When adding or modifying rules, add corresponding test cases to test/promtool/alerts_test.yaml and run:

make test-alerts

See Developer Guide for the full workflow.

Integration vs E2E Tests

Aspect Integration Tests E2E Tests
Environment Lightweight API server via envtest (no real cluster) Real Kubernetes cluster via kind
Scope Controller reconcile logic, CRD validation, status updates Full operator lifecycle including pod scheduling and networking
Speed Fast (seconds) Slower (minutes, includes cluster setup)
Framework Ginkgo + envtest kuttl