perf(scripts): scope, batch, and deduplicate markdown link checks

[Jamie Kim (jkim323)]

Pull Request

Description

The Markdown Link Check job previously revalidated the entire repository on every pull request, started one markdown-link-check Node process per Markdown file, and fetched the same external URL again whenever it appeared in another file.

This branch applies three complementary optimizations:

Changed-file pull request scoping. Markdown-Link-Check.ps1 accepts -ChangedFilesOnly and -BaseBranch, intersects the shared Git changed-file set with the existing discovery filters, and keeps full local scans as the default. Pull request validation checks changed Markdown files only, while weekly-validation.yml runs the full repository sweep.

Bounded multi-file CLI batches. Markdown targets are sorted into balanced batches capped by ThrottleLimit (default 8). Each worker invokes the pinned CLI once for multiple files, and serial JUnit conversion preserves deterministic per-file results, annotations, and fail-closed report handling.

Run-scoped external URL deduplication. Source files are first checked with external requests suppressed. Exact external URLs are then partitioned into bounded synthetic aggregates, checked once through the original configuration, and replayed to every source occurrence. Query and fragment variants remain distinct, ignored URLs remain unfetched, malformed aggregate evidence fails only mapped source files, and one temporary workspace owns all derived files and cleanup.

Bot-protected host handling. The link-check configuration now ignores https://nvd.nist.gov, which is reachable but rejects automated clients, so full local scans no longer report that host as dead.

Measured impact

Measure	Before	After	Reduction
Full-sweep Node process starts	1,122	At most 16 across source and external stages	98.6%
Full-sweep external checks	620 file-level occurrences	408 unique exact URLs	34.2%
Pull request file scope	Entire repository	Changed Markdown files only	Depends on PR; typically over 99% for small changes

A network-free local benchmark measured approximately 304 ms per isolated CLI startup. Eliminating 1,106 starts saves about 5 minutes 36 seconds per full sweep from startup overhead alone, before accounting for avoided network requests and parallel execution.

Behavior preservation

• An empty changed set reports a clean run; an unscoped scan finding nothing still fails.
• Missing, malformed, duplicate, unexpected, or incomplete JUnit attribution fails closed.
• Broken-link annotations, result JSON keys and counts, step summaries, and sorted source output remain source-file scoped.
• npm run lint:md-links still performs a full local scan by default.
• No dependency, public configuration field, or workflow permission was added.

Related Issue(s)

Fixes #2661

Fixes #2650

Type of Change

Select all that apply:

Code & Documentation:

• Bug fix (non-breaking change fixing an issue)
• New feature (non-breaking change adding functionality)
• Breaking change (fix or feature causing existing functionality to change)
• Documentation update

Infrastructure & Configuration:

• GitHub Actions workflow
• Linting configuration (markdown, PowerShell, etc.)
• Security configuration
• DevContainer configuration
• Dependency update

AI Artifacts:

• Reviewed contribution with hve-builder and addressed all actionable findings
• Copilot instructions (.github/instructions/*.instructions.md)
• Copilot prompt (.github/prompts/*.prompt.md)
• Copilot agent (.github/agents/*.agent.md)
• Copilot skill (.github/skills/*/SKILL.md)
• Copilot hook (.github/hooks/*/*.json)
• Eval spec added/updated for changed AI artifacts (evals/)

Other:

• Script/automation (.ps1, .sh, .py)
• Other (please describe):

Testing

Check	Status	Evidence
npm run test:ps -- -TestPath "scripts/tests/linting/Markdown-Link-Check.Tests.ps1"	Passed	54 selected tests passed, 0 failed; 4 Integration-tagged tests were excluded by the repository runner
PSScriptAnalyzer 1.25.0 with repository settings	Passed	Both changed PowerShell files reported 0 findings
Editor diagnostics	Passed	No diagnostics in either changed PowerShell file
git diff --check and exact source-scope inspection	Passed	No whitespace errors; implementation remained within the approved source and test files
Targeted NVD link validation	Passed	https://nvd.nist.gov was reported as ignored with the updated configuration

A post-fix full scan reported 0 broken links across 4,064 checked links. The exact npm run validate:local aggregate remains blocked on Windows because the current default source batches exceed the command-line limit; this is separate from the NVD configuration fix.

Coverage includes changed-file discovery, balanced batches, per-file JUnit attribution, exact URL deduplication, query and fragment variants, ignored URLs, shared dead-link replay, property-level aggregate trust, scoped aggregate failures, empty changed sets, temporary cleanup, and configs without ignorePatterns.

Checklist

Required Checks

• Documentation is updated (if applicable)
• Files follow existing naming conventions
• Changes are backwards compatible (if applicable)
• Tests added for new functionality (if applicable)

AI Artifact Contributions

• Used hve-builder review mode to review contribution
• Addressed all actionable findings from the hve-builder review
• Verified contribution follows common standards and type-specific requirements

Required Local Checks

The following local-safe validation commands must pass before merging:

• Local validation aggregate: npm run validate:local
• Documentation validation (if docs changed): npm run validate:docs
• Spell checking: npm run spell-check
• Link validation: npm run lint:md-links

Security Considerations

• This PR does not contain any sensitive or NDA information
• Any new dependencies have been reviewed for security issues
• Security-related scripts follow the principle of least privilege

No new dependencies were added. Workflow permissions remain contents: read, checkout retains persist-credentials: false, and full history is fetched only so changed-file merge-base resolution works.

Additional Notes

Reviewer attention is most useful on:

1. Coverage timing: pull requests validate changed Markdown files; the weekly job preserves whole-repository coverage.
2. Exact URL identity: query and fragment variants intentionally remain distinct because configuration rules can depend on them.
3. Conservative failures: untrusted source batches fail their expected files, while untrusted external aggregates fail only files mapped to URLs in that aggregate.
4. Remaining follow-up: timeout and aliveStatusCodes tuning for bot-hostile hosts remains separate policy work requiring host-specific evidence.

[github-actions]

Eval Execution

⚠️ No eval summary was produced.

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

❌ Patch coverage is 83.87097% with 60 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.61%. Comparing base (6f22d31) to head (d1c2ac9).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
scripts/linting/Markdown-Link-Check.ps1	83.87%	60 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2660      +/-   ##
==========================================
- Coverage   83.17%   81.61%   -1.56%     
==========================================
  Files         180      150      -30     
  Lines       32201    22141   -10060     
  Branches       25        0      -25     
==========================================
- Hits        26782    18070    -8712     
+ Misses       5416     4071    -1345     
+ Partials        3        0       -3     

Flag	Coverage Δ
docusaurus	?
pester	83.77% <83.87%> (-0.44%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
scripts/linting/Markdown-Link-Check.ps1	84.74% <83.87%> (+21.39%)	⬆️

... and 50 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Katrien De Graeve (katriendg)]

Great one, thanks for the improvement!

Approving, but leaving one thought and suggestion for you: fall back to a full scan for deletions and renames

The changed-Markdown scope can miss broken links when a PR deletes or renames a target, such as an image, script, template, PDF, or another Markdown file, without modifying the Markdown documents that reference it.

How does this look like:

• Continue scanning only changed Markdown files for ordinary PRs.
• If the diff contains any deletion or rename, run the full Markdown link scan.

Deletions and renames are relatively uncommon and are the changes most likely to invalidate links from otherwise unchanged documents. This preserves the performance benefit for most PRs without requiring a complex reverse-reference index that could miss encoded, reference-style, or generated links.

[Jamie Kim (jkim323)]

Great one, thanks for the improvement!

Approving, but leaving one thought and suggestion for you: fall back to a full scan for deletions and renames

The changed-Markdown scope can miss broken links when a PR deletes or renames a target, such as an image, script, template, PDF, or another Markdown file, without modifying the Markdown documents that reference it.

How does this look like:

• Continue scanning only changed Markdown files for ordinary PRs.
• If the diff contains any deletion or rename, run the full Markdown link scan.

Deletions and renames are relatively uncommon and are the changes most likely to invalidate links from otherwise unchanged documents. This preserves the performance benefit for most PRs without requiring a complex reverse-reference index that could miss encoded, reference-style, or generated links.

Thanks Katrien De Graeve (@katriendg) for the thought! Verified the gap is present. The changed set is built with --diff-filter=ACMR and FileExtensions '*.md', so a deleted or renamed .png, .pdf, .ps1, or template never enters scope, while the shared config still resolves relative targets that carry a file extension. A PR can break an unchanged document's links today.

I'd like to propose a new suggestion, because measuring the cost changed my mind about where the scoping belongs.

This PR already runs two passes with very different cost profiles

Pass	Work	Cost driver
Source	Injects ^https?:// into ignorePatterns, so it resolves local links only — zero network	file count
External	Checks the deduplicated unique URL set exactly once via synthetic files	unique URLs, network, 20s timeout, 429 retries

Changed-file scoping buys almost nothing on the first and nearly everything on the second. Measured on this branch, a full-repo source pass is 4.0s (1060 files, 2949 local links, 8-way parallel, no network). The external pass is the ~408 unique URLs already quoted in the PR description.

Suggestion: scope by link class rather than by change scope

• Always run the source pass repo-wide.
• Apply -ChangedFilesOnly to the external pass only.

The reasoning is causality: a PR can break an unchanged document's internal links by deleting, renaming, or moving a target. It cannot break that document's external links — those rot on the internet's schedule, which the weekly full scan already owns.

Compared with escalating on deletion or rename, this:

• covers internal links on every PR, not only when the heuristic fires;
• depends on no diff heuristics (--diff-filter, rename similarity, file-type coverage);
• has no bimodal worst case — under escalation, any deletion or rename PR pays the full external network scan;
• is less code: two scope variables and a filter, versus a detection probe plus escalation plus tests.

It fully covers the cases you listed, since image, script, template, PDF, and Markdown are all internal targets.

Two pre-existing issues surfaced while measuring

1. pathname:// links fail the checker. Four errors whose targets exist on disk, in docs/agents/project-planning/README.md, docs/agents/project-planning/arch-diagram-builder.md, and docs/customization/skills.md. All three files already carry these links on main, so the weekly full-scan lane is likely red today and the changed-only gate hides it. Needs ^pathname:// in markdown-link-check.config.json.

2. Split-MarkdownTargetBatch uses a fixed batch count, so batch size grows with file count. At full-repo scope on Windows, 4 of 8 batches exceed the 8191-character command-line limit (measured 8195 / 9097 / 9112 / 9761) and fail with The command line is too long. It fails closed, so it is a confusing error rather than data loss — but it already affects local npm run lint:md-links on Windows, and it blocks any repo-wide pass. Length-budgeted batching fixes it.

What are your thoughts?

[Katrien De Graeve (katriendg)]

Thanks Jamie Kim (@jkim323) for the new scoping and suggestions, I really like the approaches of scoping to local and external links, and batching.

My recommendation is to implement both before merging:

• repository-wide internal-link validation on every PR;
• changed-Markdown external-link validation;
• length-budgeted batches, including a Windows boundary test.

Please ping if you need a re-review/approval.