[prompt-analysis] Copilot PR Prompt Analysis - 2026-09-29

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-09-29

Summary

Analysis Period: Last 30 days
Total PRs: 561 | Merged: 433 (77.2%) | Closed: 125 (22.3%) | Open: 3

Merge rate among completed PRs: 77.6%. No PRs had empty bodies. Categories are assigned by title keywords.

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Bug Fix	111	97	87.4%
Feature Addition	109	68	62.4%
Refactoring	15	13	86.7%
Documentation	17	10	58.8%
Testing	13	9	69.2%
Other	296	236	79.7%

Prompt Analysis

✅ Successful Prompt Patterns

• Average body length: ~188 words
• 136 of 433 merged PRs (31%) reference concrete files or paths, compared with 66% of closed PRs (see the caveat below)
• Common title keywords: workflow, logs, github, daily, model, safe, output, failing
• Action verbs: fix, preserve, allow, infer, enable
• Examples:
  1. PR Enable narrowed label schemas in triage workflows #64266: Enable narrowed label schemas in triage workflows → Merged
  2. PR Allow workflows to narrow add-labels item schemas #64179: Allow workflows to narrow add-labels item schemas → Merged
  3. PR Infer Copilot wire API for utility model variants #64177: Infer Copilot wire API for utility model variants → Merged

❌ Unsuccessful Prompt Patterns

• Average body length: ~362 words, about twice that of merged PRs
• Common title keywords: operational (60), value (60), blocked (27), record (26), grader (24), study (23)
• 60 of the 125 closed PRs (48%) are "Record operational value study for ..." PRs, and none of them merged. This is a batch of near-identical generated tasks that were consistently rejected.
• Excluding that family, the merge rate is 433/(433+65) ≈ 87%.
• Other closures are mostly feature PRs that change behavior broadly, e.g. removing defaults or adding new commands.
• Examples:
  1. PR Manage self-hosted runner enforcement with gh aw env #64268: Manage self-hosted runner enforcement with gh aw env → Closed
  2. PR Remove implicit model defaults from fallback chains #64172: Remove implicit model defaults from fallback chains → Closed
  3. PR Improve AWF download failure diagnostics #63318: Improve AWF download failure diagnostics → Closed

Key Insights

• Bug fixes and refactors merge at about 87%. Features merge at 62% and docs at 59%.
• The operational-value study family is the main source of failures. It has a 0% merge rate and accounts for nearly half of all closures.
• Longer bodies go with closures. Closed PRs average about 2x the words of merged ones, largely because of the study family.
• Caveat on file references: the file-reference share is measured on PR bodies, which are Copilot-written summaries and not the original prompts. It is a weak signal.

Recommendations

1. DO: Frame tasks as narrowly scoped fixes with a concrete failing behavior, such as a failing workflow run or a log excerpt.
2. DO: Break feature work into small, single-behavior PRs (for example "allow X to narrow Y").
3. AVOID: Bulk-generating operational-value study PRs unless the process has been reworked. Every one of the 60 was closed.
4. AVOID: Broad behavior removals or new commands without an agreed design.

Historical Trends

Date	PRs	Merge Rate	Top Category
2026-09-29	561	77.6%	Other (296)
2026-09-28	571	78.4%	Other (293)
2026-09-26	637	79.7%	–
2026-09-25	654	79.9%	–

Trend: The merge rate has drifted down about 2 points over 4 days. The rolling window is dropping older merged PRs while the closed operational-value PRs remain.

---

Generated by Copilot PR Prompt Analysis (Run: 36628415402)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · copilot · auto · 16.8 AIC · ⌖ 5.72 AIC · ⊞ 10.8K · ◷

• expires on Sep 30, 2026, 12:49 PM UTC-08:00