[mvaneijken]

What was missing

Every Dependabot pull request waited for a maintainer's /validate, and a NuGet bump first waited for the weekly lock sweep. There was no check of who publishes a dependency before it moved forward. This is the OmadaSqlTroubleshooter approach (Fortigi/OmadaSqlTroubleshooter#178) adapted to this repository. The key difference is that nothing merges automatically here. A maintainer still merges, and that merge is what starts release.yml, so no dependency version reaches the PowerShell Gallery without a person deciding it.

What changed

• .github/workflows/dependabot-validation.yml (new)
  • Runs on pull_request_target from main's definition. It acts only when Dependabot both opened the pull request and triggered the run, and it never checks out the pull request.
  • It evaluates the pull request against the policy and writes one comment with the verdict.
  • For an eligible pull request it either:
    • dispatches the lock sync first (NuGet bump without refreshed pins), or
    • dispatches PR Validation (pr_number).
• Policy Build/DependabotValidation.psd1. A pull request is eligible only when all of these hold:
  • every update is patch or minor;
  • the new version is at least 3 days old;
  • the source repository's OpenSSF Scorecard via deps.dev is ≥ 7.0, at most 45 days old, and scores 10 on Dangerous-Workflow;
  • every commit is verified and authored by dependabot[bot] (NuGet: plus github-actions[bot]);
  • only the ecosystem's own files changed.
• Logic Build/DependabotValidation.ps1 and entry point Build/Invoke-DependabotValidation.ps1.
  • Runs on Windows PowerShell 5.1 and PowerShell 7: Invoke-WebRequest with try/catch instead of -SkipHttpErrorCheck, JSON dates accepted as string or DateTime, the PS 5.1 ConvertFrom-Json array quirk handled, TLS 1.2 enabled, step outputs written without a BOM.
  • Strict-mode safe: optional members are read through a helper.
• dependency-lock-sync.yml
  • Commits through the REST contents API, so its commits are authored by github-actions[bot] and signed by GitHub. The previous git push as devops@fortigi.nl produced unattributed, unsigned commits.
  • After committing it dispatches the evaluation again, because a push made with GITHUB_TOKEN starts no workflow run.
  • CRLF output from Update-DependencyLock.ps1 is normalised to the repository's LF.
• pr-validation.yml: new optional pr_number input for workflow_dispatch, validated and resolved to the pull request head.
• Docs: SECURITY.md (new "Dependabot validation" section; lock-sync flow updated), CONTRIBUTING.md, and a comment in dependabot.yml.

How it was verified

• Tests/Unit/DependabotValidation.Tests.ps1: 52 tests, all network calls mocked. They pass under Pester 6.2.0 / pwsh 7.4 on Linux.
  • Not run on Windows PowerShell 5.1. I had no Windows PowerShell available and PSScriptAnalyzer was not obtainable; I reviewed the code by hand for 5.1 compatibility. The powershell leg of PR Validation is the real check.
• Build/Update-MutationConfig.ps1 -Check: up to date. The tests deliberately name no module function.
• Dry run against the open pull requests deps: Bump Selenium.WebDriver and 2 others #95, deps: Bump Selenium.WebDriver and 2 others #118 and deps: Bump Selenium.WebDriver and 2 others #120, with deps.dev and other hosts stubbed:
  • All three are correctly rejected: the existing lock-sync commits are not attributed to any GitHub account.
  • deps: Bump Selenium.WebDriver and 2 others #120 also bundles System.Runtime.CompilerServices.Unsafe 4.5.3 → 6.0.0, which is a major update.
  • After this merges, ·@·d·ependabot r·ecreate gives them a clean start.
• All workflows parse as YAML.

Judgement calls and open points

• Merge stays manual (option b from the analysis). Fully automatic merge would also need an explicit release.yml dispatch, because a merge made through GITHUB_TOKEN starts no push-triggered workflow.
• Scorecard thresholds are not calibrated against real data: deps.dev was unreachable from my environment. The comment on each pull request shows the actual scores. Microsoft.Web.WebView2 will most likely land on manual review, since deps.dev probably links it to no source repository.
• Relation to Pin every GitHub Actions reference to a commit SHA #121 (SHA pinning): this PR overlaps Pin every GitHub Actions reference to a commit SHA #121 in pr-validation.yml and dependency-lock-sync.yml, but not on the same lines. The new workflow is already SHA-pinned.
• Unverified expectation: that API commits by github-actions[bot] count as attributed for the ruleset's require_extra_approval_for_unattributed_changes.

---

Generated by Claude Code

[Copilot]

🔵 Needs a closer look

It introduces security-sensitive CI automation (pull_request_target with write permissions auto-triggering validation of third-party dependency updates) whose correctness depends on runtime behavior that cannot be fully verified here, so it warrants final human review.

1 open finding

• Add comment explaining unused ModulePath parameter · New

What changed in this PR

This PR adds reputation-gated automatic validation for Dependabot pull requests. Previously every Dependabot PR waited for a maintainer's /validate comment (and NuGet bumps additionally waited for the weekly lock sweep). The new workflow evaluates each Dependabot PR against a policy — update type, release age, OpenSSF Scorecard of the source repository, and commit provenance — and, when all conditions pass, starts PR Validation automatically (first dispatching the lock sync for NuGet bumps). Crucially, nothing merges automatically; a maintainer still merges, and that merge is what triggers release.yml. The design deliberately fails closed: any network failure, missing field, or unresolved reputation signal results in manual handling rather than auto-validation.

Changes:

• New dependabot-validation.yml workflow (safe pull_request_target pattern: checks out base, reads the PR only via API) plus Build/DependabotValidation.ps1/Invoke-DependabotValidation.ps1 logic, DependabotValidation.psd1 policy, and 52 mocked unit tests.
• dependency-lock-sync.yml now commits via the REST contents API (signed github-actions[bot] commits) and re-dispatches the evaluator, since GITHUB_TOKEN pushes trigger no workflow run.
• pr-validation.yml gains a validated pr_number dispatch input; docs updated in SECURITY.md, CONTRIBUTING.md, and dependabot.yml.

File	Description
.github/​workflows/​dependabot-validation.yml	New gatekeeper workflow; evaluates Dependabot PRs and dispatches lock-sync or PR Validation.
Build/​DependabotValidation.ps1	Core evaluation logic (provenance, reputation, scorecard, report) with PS 5.1/7 + StrictMode safety.
Build/​Invoke-DependabotValidation.ps1	Entry point: resolves PR, evaluates, writes comment + step outputs.
Build/​DependabotValidation.psd1	Policy data: allowed update types, age, scorecard thresholds, per-ecosystem files/authors.
Tests/​Unit/​DependabotValidation.Tests.ps1	52 mocked unit tests covering parsing, provenance, reputation, next-step, comment.
.github/​workflows/​dependency-lock-sync.yml	REST-API commits + re-dispatch of the evaluator; actions: write added.
.github/​workflows/​pr-validation.yml	New pr_number dispatch input, validated and resolved to PR head.
SECURITY.md /​ CONTRIBUTING.md /​ .github/​dependabot.yml	Document the new validation flow and policy.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

It automates triggering PR Validation (which loads bumped dependency binaries and runs PR build code) via a pull_request_target workflow using thresholds the author states are uncalibrated and an unverified attribution assumption, so a human should review the security design before approval.

0 open findings

1 resolved since last review

• Add comment explaining unused ModulePath parameter

Previously missed (1)

In code that hasn't changed since last review

Age-based reasons make the Dependabot comment non-deterministic

Build/​DependabotValidation.ps1:621

The docstring states the report is "Free of timestamps and run ids, so an unchanged evaluation produces an identical body and the comment is not rewritten." That holds for eligible pull requests, but the "Why" section renders reasons that embed a live, clock-derived age: the release-age reason formats $AgeDays.ToString('0.0') (line 486) and the Scorecard-age reason uses [math]::Round($Age) (line 381). For a pull request that fails on age, two evaluations of the same head at different times produce different bodies, so Set-DependabotComment will patch the comment on each re-evaluation (e.g. a Dependabot synchronize/recreate) rather than leaving it untouched. Impact is minor (the comment is edited in place, not duplicated), but the stability claim here is inaccurate for age-based failures. Consider either softening this docstring or dropping the live day-count from those reason strings.

🧠 Review effort: Balanced

[Copilot]

🔵 Needs a closer look

It is a security-sensitive change (a pull_request_target workflow with write permissions and token-scoped dispatching across multiple workflows) whose cross-version PowerShell and live deps.dev behavior the author flagged as unverified, so it needs human review.

0 open findings

🧠 Review effort: Balanced

[mvaneijken]

/validate