Skip to content

Mount secrets via projected volume into Tekton pods - #59

Merged
avasilevskii merged 9 commits into
openshift-psap:mainfrom
avasilevskii:secrets-projected-volume
Apr 28, 2026
Merged

Mount secrets via projected volume into Tekton pods#59
avasilevskii merged 9 commits into
openshift-psap:mainfrom
avasilevskii:secrets-projected-volume

Conversation

@avasilevskii

@avasilevskii avasilevskii commented Apr 27, 2026

Copy link
Copy Markdown

Copy secrets referenced by FJob from psap-secrets to work namespace (e.g. psap-automation) and mount them into the Tekton pods via projected volume.

See sample FournosJob for the reference.

Closes #14.

Summary by CodeRabbit

  • New Features

    • Secrets are now copied into the operator namespace with per-job names and automatically cleaned up via owner references
    • Secrets mounted to pipeline pods via projected volumes at /var/run/secrets/fournos/<entry-name>/
    • Secret copy operations are idempotent, treating existing copies as no-ops
  • Documentation

    • Updated operator lifecycle and secret management section to reflect new copy-and-mount behavior
  • Tests

    • Added coverage for secret copying, volume mounting, and idempotency logic

Signed-off-by: avasilev <avasilev@redhat.com>
@coderabbitai

coderabbitai Bot commented Apr 27, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

This PR implements secret copying and projected volume mounting in Fournos. Secrets from a vault-synced namespace are copied into the operator namespace with per-job naming and owner references, then mounted via a projected volume in Tekton PipelineRun pods.

Changes

Cohort / File(s) Summary
Documentation
Fournos_Design_Document.md, README.md
Updated design and README to reflect new secret-copying behavior during Admitted lifecycle, projected volume mounting at /var/run/secrets/fournos/<entry-name>/, and cleanup expectations for copied secrets with ownerReferences.
Tekton Task Mounts
config/forge/workflows/tasks.yaml, config/fournos-validation/workflows/tasks.yaml
Updated kubeconfig mount paths to /var/run/secrets/fournos-kubeconfig and added vault-secrets volume mount to /var/run/secrets/fournos (read-only) in task steps.
Mock Pipelines & Secrets
dev/mock-pipelines/tasks.yaml, dev/mock-secrets.yaml, dev/mock-resolve/resolve.sh, dev/job-secret-demo.yaml
Added demo FournosJob manifest, introduced vault-placeholder mock secret, and updated mock-resolve script to unconditionally set spec.secretRefs post-conditionals; mock-pipelines task now enumerates and logs secrets from the mounted volume.
Secret Management Core Logic
fournos/core/clusters.py, fournos/core/tekton.py, fournos/handlers/execution.py
Added ResolvedSecret dataclass and copy_secret(s) methods for copying vault-synced secrets into operator namespace with owner references; updated TektonClient to accept resolved secrets and build a projected vault-secrets volume; modified execution handler to call copy_secrets and pass resolved secrets to PipelineRun creation.
RBAC & Test Infrastructure
manifests/rbac/role_fournos.yaml, tests/conftest.py, tests/unit/conftest.py
Granted additional create verb on secrets resource; added get_pipelinerun_volumes test helper and unit-test conftest to isolate unit tests from e2e Kubernetes requirements.
Secret Handling Tests
tests/test_secret_refs.py, tests/unit/test_secret_volume.py
Expanded e2e secret test to verify secret copying and projected volume integration; added comprehensive unit tests for _build_secrets_volume, secret copy idempotency (409 conflict handling), and copy_secret(s) behavior with mocked Kubernetes interactions.

Sequence Diagram

sequenceDiagram
    participant FJ as FournosJob<br/>(Admitted)
    participant HR as Handler<br/>(reconcile_admitted)
    participant CR as ClusterRegistry<br/>(copy_secrets)
    participant K8s as Kubernetes API
    participant TC as TektonClient<br/>(create_pipeline_run)
    participant PR as PipelineRun<br/>(with projected volume)

    FJ->>HR: Trigger reconcile (Admitted state)
    activate HR
    HR->>CR: copy_secrets(secret_refs, fjob_name, owner_ref)
    activate CR
    CR->>K8s: Read secrets from secrets_namespace
    K8s-->>CR: Secret data + keys
    CR->>K8s: Create secret copy in operator namespace<br/>(name: fjob_name-ref, ownerRef: FournosJob)
    K8s-->>CR: Created Secret / 409 Conflict (idempotent)
    CR-->>HR: ResolvedSecret list (name, original_name, keys)
    deactivate CR
    HR->>TC: create_pipeline_run(..., resolved_secrets)
    activate TC
    TC->>TC: Build projected vault-secrets volume<br/>(from resolved_secrets entries)
    TC->>K8s: Create PipelineRun with podTemplate<br/>(includes projected volume)
    K8s-->>TC: PipelineRun created
    deactivate TC
    TC-->>HR: PipelineRun resource
    deactivate HR
    HR->>PR: Pods access secrets at<br/>/var/run/secrets/fournos/
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • #56: Introduces settings.secrets_namespace and foundational secret lookup changes that this PR builds upon with copy and projected-volume logic.
  • #49: Adds initial secret-resolution work that this PR extends by replacing resolve-only logic with copy/ResolvedSecret semantics and projected-volume mounting.
  • #29: Modifies TektonClient ownership handling and PipelineRun construction; this PR similarly updates create_pipeline_run signature and adds projected volume configuration in the same function.

Suggested labels

lgtm

Poem

🐰 Secrets now hop safely through the namespace lane,
Copied with care and an owner's chain,
Projected volumes mount them just right,
Tekton pods read each file in sight!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Mount secrets via projected volume into Tekton pods' directly and accurately describes the main change: implementing secret mounting via projected volumes in Tekton PipelineRuns.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci

openshift-ci Bot commented Apr 27, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 27, 2026
@openshift-ci

openshift-ci Bot commented Apr 27, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign avasilevskii for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@avasilevskii

Copy link
Copy Markdown
Author

/test fournos

avasilev added 4 commits April 28, 2026 11:24
Signed-off-by: avasilev <avasilev@redhat.com>
Signed-off-by: avasilev <avasilev@redhat.com>
Signed-off-by: avasilev <avasilev@redhat.com>
Signed-off-by: avasilev <avasilev@redhat.com>
@avasilevskii
avasilevskii marked this pull request as ready for review April 28, 2026 10:21
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 28, 2026
Signed-off-by: avasilev <avasilev@redhat.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
fournos/handlers/execution.py (1)

160-170: Use a distinct condition reason for API copy failures.

ApiException failures here are not always “not found”, but the condition reason is always set to SecretRefNotFound. Consider splitting reason values so status stays diagnosable (SecretRefNotFound vs SecretCopyFailed).

♻️ Proposed adjustment
-        except (KeyError, client.exceptions.ApiException) as exc:
-            msg = str(exc).strip("'\"") if isinstance(exc, KeyError) else exc.reason
+        except (KeyError, client.exceptions.ApiException) as exc:
+            is_not_found = isinstance(exc, KeyError)
+            msg = str(exc).strip("'\"") if is_not_found else exc.reason
+            cond_reason = "SecretRefNotFound" if is_not_found else "SecretCopyFailed"
             patch.status["phase"] = Phase.FAILED
             patch.status["message"] = msg
             set_condition(
                 patch,
                 conditions,
                 COND_PIPELINE_RUN_READY,
                 "False",
-                "SecretRefNotFound",
+                cond_reason,
                 msg,
             )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@fournos/handlers/execution.py` around lines 160 - 170, The except block
currently conflates KeyError and client.exceptions.ApiException by always using
"SecretRefNotFound" as the condition reason; change the handling so KeyError
continues to use reason "SecretRefNotFound" while ApiException uses a distinct
reason like "SecretCopyFailed" (use instanceof checks on exc or separate except
clauses), keep msg for KeyError as str(exc).strip(...) and for ApiException as
exc.reason, set patch.status["phase"]=Phase.FAILED and
patch.status["message"]=msg in both cases, and pass the appropriate reason into
the set_condition call for COND_PIPELINE_RUN_READY so callers can distinguish
missing refs from API copy failures.
tests/unit/test_secret_volume.py (1)

92-170: Add a regression test for non-vault secret rejection in copy_secret.

Current tests cover happy-path and API-error behavior, but not the label-gate path (fournos.dev/vault-entry=true) that should block non-vault secrets.

🧪 Suggested test case
 class TestCopySecret:
+    def test_rejects_secret_without_vault_label(self, registry):
+        reg, k8s = registry
+        source = _make_source_secret(user="x")
+        source.metadata = mock.MagicMock(labels={})
+        k8s.read_namespaced_secret.return_value = source
+
+        with pytest.raises(KeyError):
+            reg.copy_secret("creds", "my-job", OWNER_REF)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/unit/test_secret_volume.py` around lines 92 - 170, Add a regression
test in tests/unit/test_secret_volume.py that covers the label-gate path: have
k8s.read_namespaced_secret return a secret that does NOT include the
"fournos.dev/vault-entry":"true" label, then call reg.copy_secret("name", "job",
OWNER_REF) and assert that it raises (use pytest.raises(Exception)) and that
k8s.create_namespaced_secret was not called; reference the existing fixtures and
the reg.copy_secret, k8s.read_namespaced_secret and k8s.create_namespaced_secret
calls to mirror the other tests' structure.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@dev/mock-pipelines/tasks.yaml`:
- Around line 68-69: The current loop prints secret contents by using cat inside
the while loop (find ... | while read f; do echo "  $f = $(cat "$f")"; done);
update it to avoid printing values and instead emit only metadata—e.g., echo the
file path plus size and modification time or a presence marker (use stat or ls
-l) like: for each file referenced by the find/while construct, replace the
"$(cat "$f")" expansion with a safe metadata expression (stat -c '%n %s %y' "$f"
or a simple "exists" marker) so logs show filenames and metadata only, not
secret contents.

In `@fournos/core/clusters.py`:
- Around line 81-83: The copy_secret flow currently reads a Secret from
settings.secrets_namespace then copies it without re-checking the Vault label;
update the copy_secret implementation to re-validate that the fetched source
Secret has metadata.labels.get('fournos.dev/vault-entry') == "true" before
proceeding (after self._k8s.read_namespaced_secret(ref,
settings.secrets_namespace)); if the label is missing or not "true", abort the
copy (raise or return an error and do not use source.data). Ensure you reference
the source object returned by read_namespaced_secret and the copy_secret
function name when making the change.

---

Nitpick comments:
In `@fournos/handlers/execution.py`:
- Around line 160-170: The except block currently conflates KeyError and
client.exceptions.ApiException by always using "SecretRefNotFound" as the
condition reason; change the handling so KeyError continues to use reason
"SecretRefNotFound" while ApiException uses a distinct reason like
"SecretCopyFailed" (use instanceof checks on exc or separate except clauses),
keep msg for KeyError as str(exc).strip(...) and for ApiException as exc.reason,
set patch.status["phase"]=Phase.FAILED and patch.status["message"]=msg in both
cases, and pass the appropriate reason into the set_condition call for
COND_PIPELINE_RUN_READY so callers can distinguish missing refs from API copy
failures.

In `@tests/unit/test_secret_volume.py`:
- Around line 92-170: Add a regression test in tests/unit/test_secret_volume.py
that covers the label-gate path: have k8s.read_namespaced_secret return a secret
that does NOT include the "fournos.dev/vault-entry":"true" label, then call
reg.copy_secret("name", "job", OWNER_REF) and assert that it raises (use
pytest.raises(Exception)) and that k8s.create_namespaced_secret was not called;
reference the existing fixtures and the reg.copy_secret,
k8s.read_namespaced_secret and k8s.create_namespaced_secret calls to mirror the
other tests' structure.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ee35bfa3-10c3-42a8-8bd6-2ec31d8803f8

📥 Commits

Reviewing files that changed from the base of the PR and between 08736c2 and a77edb4.

📒 Files selected for processing (17)
  • Fournos_Design_Document.md
  • README.md
  • config/forge/workflows/tasks.yaml
  • config/fournos-validation/workflows/tasks.yaml
  • dev/job-secret-demo.yaml
  • dev/mock-pipelines/tasks.yaml
  • dev/mock-resolve/resolve.sh
  • dev/mock-secrets.yaml
  • fournos/core/clusters.py
  • fournos/core/tekton.py
  • fournos/handlers/execution.py
  • manifests/rbac/role_fournos.yaml
  • tests/conftest.py
  • tests/test_secret_refs.py
  • tests/unit/__init__.py
  • tests/unit/conftest.py
  • tests/unit/test_secret_volume.py

Comment thread dev/mock-pipelines/tasks.yaml Outdated
Comment thread fournos/core/clusters.py
avasilev added 2 commits April 28, 2026 12:29
Signed-off-by: avasilev <avasilev@redhat.com>
Signed-off-by: avasilev <avasilev@redhat.com>
@kpouget

kpouget commented Apr 28, 2026

Copy link
Copy Markdown
Collaborator

/test fournos

1 similar comment
@avasilevskii

Copy link
Copy Markdown
Author

/test fournos

@psap-forge-bot

Copy link
Copy Markdown

🟢 Test of 'fournos_deploy --project-source' succeeded after 00 hours 10 minutes 54 seconds 🟢

• Link to the test results.

• No reports index generated...

Test configuration:

/test fournos

Execution logs

@kpouget

kpouget commented Apr 28, 2026

Copy link
Copy Markdown
Collaborator

thanks @avasilevskii,
/lgtm
we'll test it further when #67 is fixed

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Apr 28, 2026
@avasilevskii
avasilevskii merged commit d6aebd5 into openshift-psap:main Apr 28, 2026
5 of 6 checks passed
@coderabbitai coderabbitai Bot mentioned this pull request Apr 28, 2026
@avasilevskii
avasilevskii deleted the secrets-projected-volume branch April 28, 2026 14:53
@coderabbitai coderabbitai Bot mentioned this pull request Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Task: allow the job to specify the secrets it needs

2 participants