🏥 Repository Health Dashboard

[github-actions]

🏥 Daily Health Check — 2026-08-20

Status: 🔴 8 critical · 🟡 2 warnings · 🔵 0 info
Since yesterday: 🆕 7 new · ✅ 3 resolved · 📌 2 unchanged

---

🆕 New Findings (7)

These appeared since the last health check (2026-08-19).

🔴 Evaluation workflow failed on main: vally (dotnet-maui--gpt-5.6-luna) — "Select available Copilot token from pool" step

• Fingerprint: pipeline:evaluation:vally-(dotnet-maui--gpt-5.6-luna):select-available-copilot-token-from-pool:failure
• Details: Run #32226917771 (schedule: default, schedule event on main, created 2026-08-19T07:14:30Z) failed in job evaluate / vally (dotnet-maui--gpt-5.6-luna) at the Select available Copilot token from pool step.
• Action: Inspect the Copilot PAT pool — token exhaustion/rate limiting is preventing this shard from acquiring a usable token.

🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-b--claude-sonnet-4.6) — "Select available Copilot token from pool" step

• Fingerprint: pipeline:evaluation:vally-(dotnet-test--shard-b--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
• Details: Run #32279143923 (issue_comment on main, created 2026-08-19T17:00:25Z) failed in job evaluate / vally (dotnet-test--shard-b--claude-sonnet-4.6) at the Select available Copilot token from pool step.
• Action: Same PAT-pool exhaustion pattern as the other shard failures below — investigate pool capacity/rotation.

🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-c--gpt-5.6-luna) — "Select available Copilot token from pool" step

• Fingerprint: pipeline:evaluation:vally-(dotnet-test--shard-c--gpt-5.6-luna):select-available-copilot-token-from-pool:failure
• Details: Run #32281754879 (issue_comment on main, created 2026-08-19T17:28:01Z) failed in job evaluate / vally (dotnet-test--shard-c--gpt-5.6-luna) at the Select available Copilot token from pool step.
• Action: Same PAT-pool exhaustion pattern — see correlation note below.

🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-default--claude-sonnet-4.6) — "Select available Copilot token from pool" step

• Fingerprint: pipeline:evaluation:vally-(dotnet-test--shard-default--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
• Details: Failed in 3 separate runs today: Run #32281754879, Run #32279143923, Run #32278034405 — all on main, all at evaluate / vally (dotnet-test--shard-default--claude-sonnet-4.6) / Select available Copilot token from pool.
• Action: Highest-recurrence shard for this failure mode today — prioritize PAT pool investigation.

🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-default--gpt-5.6-luna) — "Select available Copilot token from pool" step

• Fingerprint: pipeline:evaluation:vally-(dotnet-test--shard-default--gpt-5.6-luna):select-available-copilot-token-from-pool:failure
• Details: Run #32281833213 (workflow_dispatch on main, created 2026-08-19T17:28:53Z) failed in job evaluate / vally (dotnet-test--shard-default--gpt-5.6-luna) at the Select available Copilot token from pool step.
• Action: Same PAT-pool exhaustion pattern.

🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-risk-coverage--claude-sonnet-4.6) — "Select available Copilot token from pool" step

• Fingerprint: pipeline:evaluation:vally-(dotnet-test--shard-risk-coverage--claude-sonnet-4.6):select-available-copilot-token-from-pool:failure
• Details: Failed in 2 runs today: Run #32281754879, Run #32279143923.
• Action: Same PAT-pool exhaustion pattern.

🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-risk-coverage--gpt-5.6-luna) — "Select available Copilot token from pool" step

• Fingerprint: pipeline:evaluation:vally-(dotnet-test--shard-risk-coverage--gpt-5.6-luna):select-available-copilot-token-from-pool:failure
• Details: Failed in 2 runs today: Run #32281754879, Run #32279143923.
• Action: Same PAT-pool exhaustion pattern.

---

🔍 Investigation Results

Deep investigations are dispatched for new critical/warning findings.
The grooming workflow links results ~3 hours after this run.

Finding	Severity	Investigation	First Seen	Result
Evaluation workflow failed on main: vally (dotnet-maui--gpt-5.6-luna) — "Select available Copilot token from pool" step	🔴 Critical	🔄 Dispatched	2026-08-20	⏳ Investigation dispatched — results arriving shortly...
Evaluation workflow failed on main: vally (dotnet-test--shard-b--claude-sonnet-4.6) — "Select available Copilot token from pool" step	🔴 Critical	🔄 Dispatched	2026-08-20	⏳ Investigation dispatched — results arriving shortly...

---

✅ Resolved Since Yesterday (3)

These were in yesterday's report but are no longer detected.

Evaluation average duration exceeds 55-minute threshold

Sample of the 5 most recent completed main runs today averages ~22.1 min — well under the 50/55-minute thresholds (previously ~56.5 min). Continue monitoring, as the sample size is small.

DevOps Health — Groom Dashboard workflow failed: Execute GitHub Copilot CLI step

No recurrence of this failure in the last 24h scan of main failed runs.

DevOps Daily Health Check workflow failed: Process Safe Outputs step

No recurrence of this failure in the last 24h scan of main failed runs.

---

📌 Existing Findings (2)

These have been present since before today. Sorted by age.

🟡 Warning — Orphan plugin: dotnet-experimental not in marketplace.json · first seen 2026-05-14 · 65 occurrences

Fingerprint: infra:orphan-plugin:dotnet-experimental
Category: Infra · Severity: 🟡 Warning

The plugin directory plugins/dotnet-experimental/ has a valid plugin.json but is still not listed in .github/plugin/marketplace.json. Consumers cannot discover this plugin. ~98 days outstanding.

Links: plugin.json · marketplace.json

Suggested action: Either add dotnet-experimental to marketplace.json if it is ready for consumers, or remove the plugin directory if it is no longer needed.

🟡 Warning — Validate PAT Pool workflow failed: Build summary step · first seen 2026-08-05 · 6 occurrences

Fingerprint: pipeline:validate-pat-pool:validate-copilot-pat-pool:build-summary:failure
Category: Pipeline · Severity: 🟡 Warning

The validate-pat-pool.yml workflow failed again at the Build summary step (Run #32325482086, 2026-08-20T02:40:39Z, scheduled). This recurs intermittently alongside today's Copilot token-pool exhaustion failures in the evaluation workflow — likely the same underlying PAT pool capacity/health issue.

Links: validate-pat-pool.yml runs

Suggested action: Correlate this recurring failure with the new evaluation PAT-pool exhaustion findings above — both point to the same Copilot token pool health problem.

---

📊 Trends (7-day)

Metric	Today	7d Avg	Δ	Trend
Eval duration (min)	~22.1 (5 runs sampled, main)	~56.5 (prior run)	-34.4	✅
Eval success rate (main, sample)	0% (0/5 success+failure)	40% (prior)	-40pp	⚠️
Eval success rate (all branches, 24h)	61.5% (8/13 success+failure)	89% (prior)	-27.5pp	⚠️
Eval scheduled cancellation rate (24h)	0% (0 scheduled runs on main in window)	0% (prior)	0	➡️
Workflow failure rate (7d)	9 active pipeline/resource fingerprints (8 critical, 1 warning)	4 (prior)	+5	⚠️
Compute hours/day	N/A (not computed this run)	N/A	—	➡️

Correlation: All 7 new critical findings share the same root failure — the "Select available Copilot token from pool" step across multiple vally evaluation shards — pointing to a systemic Copilot PAT pool exhaustion issue, not isolated flakes. This correlates directly with the existing validate-pat-pool Build-summary failure (also PAT-pool related). Overall eval failure rate across all branches in 24h is 38.5% (5 failures / 13 completed), which exceeds the 30% critical threshold (fingerprint pipeline:evaluation:failure-rate:critical, unchanged from yesterday, now 2 occurrences).

Recommendations:

1. Investigate Copilot PAT pool capacity/rotation — it is the common cause behind 7 new critical findings plus the recurring validate-pat-pool warning.
2. Consider increasing the PAT pool size or improving token rotation/backoff logic in the "Select available Copilot token from pool" step.
3. Re-verify the eval-duration improvement (~22 min today) once a larger sample accumulates — today's sample is small (5 runs) and may not be representative.

⚠️ Skipped Pages deployment check (I5): no read-only tool available in this environment to query repository Pages deployment status without an authenticated gh session.

---

_{🤖 Generated by DevOps Health Check agentic workflow · Run #32327839808 · 2026-08-20T03:21:26 UTC}> Generated by DevOps Daily Health Check · auto · 160.9 AIC · ⌖ 15.2 AIC · ⊞ 20.5K · ◷

[github-actions]

📋 Health Check — 2026-08-12

🆕 4 new · ✅ 3 resolved · 📌 2 unchanged

New:

• 🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-b) — Select available Copilot token from pool (run)
• 🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-c) — Select available Copilot token from pool (run)
• 🟡 pr-triage workflow failed: Run pr-triage-act.sh step (run)
• 🟡 Evaluation failure rate across all branches is 30% (last 24h)

Resolved:

• Evaluation failure rate across all branches exceeds 30% (critical bucket) — now at exactly 30%, moved to warning bucket
• DevOps Health — Groom Dashboard workflow failed: Execute GitHub Copilot CLI step — last run succeeded
• DevOps Daily Health Check workflow failed: Process Safe Outputs step — last run succeeded

2 investigations dispatched (both critical, budget cap reached): shard-b and shard-c Copilot-token-pool failures.

⚠️ Note: cache-memory writes were denied by sandbox policy this run (reads worked fine) — the fingerprint/history state could not be persisted for next run's diff base. Reported separately via missing_tool.

Full report →

Generated by DevOps Daily Health Check · auto · 137.9 AIC · ⌖ 4.22 AIC · ⊞ 20.3K · ◷

[github-actions]

📋 Health Check — 2026-08-13

🆕 1 new · ✅ 3 resolved · 📌 3 unchanged

New:

• 🔴 Evaluation workflow failed on main: vally (dotnet-test) — "Run vally evaluations" step (run)

Resolved:

• Evaluation failure rate across all branches exceeds 30% (now ~11%)
• DevOps Health — Groom Dashboard workflow failed
• DevOps Daily Health Check workflow failed

Full report →

Generated by DevOps Daily Health Check · auto · 163.9 AIC · ⌖ 9.88 AIC · ⊞ 20.3K · ◷

[github-actions]

📋 Health Check — 2026-08-20

🆕 7 new · ✅ 3 resolved · 📌 2 unchanged

New:

• 🔴 Evaluation workflow failed on main: vally (dotnet-maui--gpt-5.6-luna) — Copilot token pool
• 🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-b--claude-sonnet-4.6) — Copilot token pool
• 🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-c--gpt-5.6-luna) — Copilot token pool
• 🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-default--claude-sonnet-4.6) — Copilot token pool (3 recurrences today)
• 🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-default--gpt-5.6-luna) — Copilot token pool
• 🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-risk-coverage--claude-sonnet-4.6) — Copilot token pool (2 recurrences today)
• 🔴 Evaluation workflow failed on main: vally (dotnet-test--shard-risk-coverage--gpt-5.6-luna) — Copilot token pool (2 recurrences today)

All 7 new criticals share the same root cause: Copilot PAT pool exhaustion at the "Select available Copilot token from pool" step. 2 investigations dispatched (budget cap).

Resolved:

• Evaluation average duration exceeds 55-minute threshold (now ~22 min avg on small sample)
• DevOps Health — Groom Dashboard workflow failed: Execute GitHub Copilot CLI step
• DevOps Daily Health Check workflow failed: Process Safe Outputs step

Full report →

Generated by DevOps Daily Health Check · auto · 160.9 AIC · ⌖ 15.2 AIC · ⊞ 20.5K · ◷