[skill-optimizer] Daily Skill Optimizer Improvements - 2026-08-18

[github-actions]

Summary

• Run mode: dry-run
• Status: ✅ (exit code 0)

Key Findings

1. Provision OPENROUTER_API_KEY as a repo/org secret so the optimizer can run in benchmark mode. Expected impact: enables real skill-quality scoring instead of a no-op dry-run every day, turning this workflow into an actual quality signal rather than a log-only placeholder.
2. Add a fallback/notification step when the suite is skipped (no API key) so silent skips are visible. Expected impact: prevents weeks of "green but empty" runs from masking the fact that no analysis is happening; surfaces the gap to maintainers immediately via a clearer log line or workflow annotation.
3. Trim/cache the npm ci dependency install (362 packages, 19 vulnerabilities reported) in the skill_optimizer job. Expected impact: faster job runtime and reduced exposure to flagged high-severity vulnerabilities in the fetched fastxyz/skill-optimizer toolchain; consider pinning a vetted version and enabling npm cache/actions/cache for the clone+install step.

Evidence from Artifact

• summary.json: {"repository":"github/gh-aw","run_mode":"dry-run","run_status":0,"run_url":"https://github.com/github/gh-aw/actions/runs/32093562566"}
• run.log: dry-run: Docker available but OPENROUTER_API_KEY not set; skipping suite execution — confirms the benchmark suite never actually executed, so no real skill-quality metrics were produced this run.
• npm-ci.log: added 362 packages, and audited 363 packages in 9s followed by 19 vulnerabilities (2 low, 6 moderate, 11 high) — dependency install is non-trivial and carries known vulnerabilities in the optimizer's own toolchain.
• npm-build.log: standard tsc build succeeded with no errors, so the tool itself builds fine — the only gap is the missing API key blocking real analysis.
• No suite-results/ directory present in the artifact, consistent with run_mode=dry-run (would only appear for run_mode=benchmark).

Recommendations

1. Add OPENROUTER_API_KEY as an encrypted secret and switch this workflow to run_mode=benchmark (at least on a scheduled cadence) so the skill optimizer produces actionable scoring instead of a permanent dry-run.
2. Emit a clear workflow-level warning/annotation (not just a log line) when the suite is skipped due to a missing API key, so the no-op state is visible in the Actions UI summary rather than buried in run.log.
3. Pin fastxyz/skill-optimizer to a specific vetted commit/tag and cache its node_modules (or use npm ci --prefer-offline with actions/cache) to cut install time and reduce exposure to the 11 high-severity vulnerabilities currently reported by npm audit.

Generated by ⚡ Daily Skill Optimizer Improvements · auto · 9.98 AIC · ⌖ 4.12 AIC · ⊞ 6.8K · ◷

• expires on Aug 24, 2026, 7:00 PM UTC-08:00

[pelikhan]

@copilot delete this agentic workflow