[prompt-analysis] Copilot PR Prompt Analysis - 2026-09-25

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-09-25

Summary

Analysis Period: Last 30 days
Total PRs: 654 | Merged: 520 (79.5%) | Closed: 131 (20.0%) | Open: 3 (0.5%)
Merge rate (of completed PRs): 79.9%

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Bug Fix	282	221	60	78.6%
Feature Addition	186	147	38	79.5%
Testing	68	60	7	89.6%
Documentation	33	27	6	81.8%
Refactoring	1	1	0	100.0% (n=1, not significant)
Other	84	64	20	76.2%

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 124.5 words
• Prompts containing a code block/snippet: 79.0% of merged PRs
• Most common keywords: coverage, when, changes, add, existing, before
• Testing-labeled prompts had the highest success rate (89.6%)

Example successful prompts:

1. PR [ci-coach] Speed up slow rate-limit sleep in cgo A-C unit-test shard #63187: "CI Optimization Proposal ### Summary Pre-flight validation (deps-dev, lint, lint-errors, npm-ci..." → Merged
2. PR [test-parallel] Add t.Parallel() to safe Go test candidates (episode, fallback integration, filtering) #56216: "Analyzed 25 files (batch starting after pkg/cli/lint_command_test.go, round-robin position ends..." → Merged
3. PR Fix --count/--timeout wiring for multi-target logs downloads #60308: "A multi-target gh aw logs run downloads a large number of runs but reports only a handful in the final table..." → Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 130.8 words (only slightly longer than merged — length alone is not the driver)
• Prompts containing a code block/snippet: only 62.6% of closed PRs (16pt lower than merged)
• Most common keywords: evidence, operational, value, study, output, file
• Prompts referencing "operational value" grading/evidence work had a markedly lower success rate: 21.5% (17 merged / 62 closed out of 79)

Example unsuccessful prompts:

1. PR Add an opt-in shell-free go-repository tool profile for Copilot SDK workflows #62412: "Add an opt-in shell-free go-repository tool profile for Copilot SDK workflows..." → Closed
2. PR [WIP] Add operational-value grader for code scanning fixer #58521: "[WIP] Add operational-value grader for code scanning fixer..." → Closed
3. PR Fix operational-value grader created-at acquisition and preserve grader result contract in gh aw logs #60635: "Fix operational-value grader created-at acquisition and preserve grader result contract in gh aw logs..." → Closed

Key Insights

• Pattern 1: Prompts that include a concrete code block, YAML/JSON snippet, or file path reference merge 79.0% of the time vs. 62.6% for prompts without one — concrete artifacts correlate with higher success.
• Pattern 2: Prompts touching "operational-value" grading/evidence infrastructure have a notably low success rate (21.5%), consistent with prior days' data (this pattern has recurred for several days running), suggesting this subsystem carries elevated review/rework risk.
• Pattern 3: Testing-category prompts ("add test", "test coverage", "regression") continue to have the highest success rate (89.6%), a trend consistent with the last several days (89–100% range), reinforcing that well-scoped, verification-focused asks merge reliably.
• Pattern 4: Overall prompt length between merged (124.5 words) and closed (130.8 words) prompts is similar — conciseness alone doesn't explain outcomes here; specificity (code references, concrete asks) matters more than word count.

Recommendations

Based on today's analysis:

1. DO: Include a concrete code snippet, config block, or specific file/function reference in the prompt — these correlate with a ~16-point higher merge rate.
2. DO: Frame requests around adding/improving test coverage or fixing a specific, reproducible bug — these categories have the highest success rates.
3. AVOID: Broad "operational value"/evidence/grading-related asks without a narrow, verifiable scope — this category has consistently underperformed across multiple days of analysis.

Historical Trends

Date	PRs	Success Rate	Top Category
2026-09-25	654	79.9%	refactor*
2026-09-24	726	80.4%	testing
2026-09-23	763	81.6%	testing
2026-09-22	797	82.1%	testing

*Refactor category had only 1 sample today (100% rate), not statistically meaningful; testing (89.6%) is the most reliable top performer.

Trend: The overall merge rate has declined slightly over the last four days (82.1% → 79.9%), alongside a shrinking total PR volume (797 → 654), possibly reflecting a slower period or a shift toward more complex/riskier changes. The "operational-value" pattern's weak success rate has now appeared in the data for multiple consecutive days, suggesting it's a durable signal rather than noise.

---

Generated by Copilot PR Prompt Analysis (Run: 36187333913)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · copilot · auto · 38.6 AIC · ⌖ 9.63 AIC · ⊞ 10.9K · ◷

• expires on Sep 26, 2026, 12:48 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Copilot PR Prompt Pattern Analysis.

A newer discussion is available at Discussion #63675.