[prompt-analysis] Copilot PR Prompt Analysis - 2026-08-05

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis — 2026-08-05

Analysis Period: Last 30 days  |  Total PRs: 1000  |  Merged: 746 (74.6%)  |  Closed: 248 (24.8%)  |  Open: 6 (0.6%)

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Testing	23	20	3	87.0%
Other (no clear verb)	373	281	88	76.2%
Documentation	44	33	11	75.0%
Bug Fix	326	246	80	75.5%
Feature Addition	167	120	45	72.7%
Refactoring	67	46	21	68.7%

Categorization is based on the PR title's leading verb (fix/add/refactor/document/test); most PR bodies follow a standard template, so title phrasing is the strongest categorical signal.

Prompt length and keyword analysis

Average prompt length (words, after stripping URLs/badges/code fences):

• Merged: 114.1 words (n=746)
• Closed: 123.6 words (n=248)
• Open: 72.3 words (n=6)

Closed PRs skew slightly longer than merged ones — verbosity alone isn't the differentiator; content matters more.

Notable keyword signal — CVE/security-advisory PRs: 15 of 248 closed PRs (6%) reference CVE identifiers (dependency/security-tracker bumps), vs. only 1 of 746 merged PRs. These automated security-scan PRs appear to close at a much higher rate than typical feature/bugfix work, likely due to false positives, duplicate advisories, or being superseded by other dependency updates.

Top keywords in merged PR bodies (post stop-word filtering): engine, fixes, start, suffix, coding, added, changes, details, summary, tests, behavior, coverage, detection, failure, error

Top keywords in closed PR bodies: issue, fixes, start, suffix, engine, headers, description, comments, section, prompt, image, container

✅ Successful Prompt Examples

1. PR Fix Metrics Collector zeroing safe-output metrics on logs tool timeout #50576 — Fix Metrics Collector zeroing safe-output metrics on logs tool timeout → Merged. Names the exact component, the specific failure mode, and root cause ("60s tool timeout").
2. PR fix: add missing Kiro network domains and SECRET_ auth key fallbacks #50573 — fix: add missing Kiro network domains and SECRET_ auth key fallbacks → Merged. Conventional-commit style title, lists concrete missing items (domains, auth fallback).
3. PR feat: migrate 6 daily workflows to opencode, crush, and goose engines #50557 — feat: migrate 6 daily workflows to opencode, crush, and goose engines → Merged. Precise scope ("6 daily workflows") and named targets (engines).

❌ Unsuccessful Prompt Examples

1. PR Document gh aw pr command in CLI reference #50577 — Document gh aw pr command in CLI reference → Closed. Documentation PRs with narrow, single-file scope sometimes get superseded/rejected in favor of manual doc edits.
2. PR [WIP] Fix Kiro CLI auth failures due to network allow-list gap #50555 — [WIP] Fix Kiro CLI auth failures due to network allow-list gap → Closed. The [WIP] marker and checklist-style body (partially checked) suggest the task was abandoned or superseded by PR Fix intermittent Kiro/Cursor auth by updating Kiro egress allow-list and SECRET_* API key fallback #50565, a cleaner duplicate of the same fix.
3. PR eslint-factory: detect ternary-wrapped env numeric parse assignments in NaN-check rule #50512 — eslint-factory: detect ternary-wrapped env numeric parse assignments in NaN-check rule → Closed. Highly narrow/edge-case lint rule; may indicate scope was too speculative/low-value to merge.
4. PR Improve docs accessibility by ensuring skip-navigation targets the main landmark #50511 — Improve docs accessibility by ensuring skip-navigation targets the main landmark → Closed. Vague improvement framing ("Improve") without a concrete bug report driving it.

Key Insights

• Refactoring has the lowest success rate (68.7%) among clearly-categorized types — open-ended "improve/optimize" prompts without a concrete failing behavior appear more likely to be rejected or superseded than prompts anchored to a specific bug or feature request.
• CVE/security-advisory-driven PRs are disproportionately closed (15/16 CVE-referencing PRs, 94%, were closed vs. an average 25% close rate) — these are typically auto-generated dependency-update PRs that get superseded, duplicated, or deemed false positives rather than reflecting poor prompt quality.
• Duplicate/parallel prompts reduce success: multiple PRs targeting the same root cause (e.g., [WIP] Fix Kiro CLI auth failures due to network allow-list gap #50555 vs Fix intermittent Kiro/Cursor auth by updating Kiro egress allow-list and SECRET_* API key fallback #50565, both "Fix Kiro CLI auth") show that once a fix lands via one PR, competing PRs addressing the identical issue are closed — suggesting prompt dedup or coordination could reduce wasted PRs.
• Testing-focused prompts have the highest success rate (87%), likely because test additions are low-risk, self-contained, and rarely conflict with other in-flight changes.

Recommendations

1. DO: Anchor prompts to a specific failing behavior, error message, or file/function reference (as in the "Fix Metrics Collector..." example) rather than a general improvement goal.
2. DO: Use conventional-commit-style, scoped titles (fix:, feat:) that name the exact component and blast radius.
3. AVOID: Open-ended "improve/optimize/refactor" prompts without a concrete triggering issue — these had the lowest merge rate (68.7%).
4. AVOID: Launching duplicate/parallel Copilot tasks for the same root-cause bug — check for in-flight PRs first to prevent redundant closures.

Historical Trends (last 8 days)

Date	PRs	Success Rate	Top Category
2026-08-05	1000	75.1%	Testing (87.0%)
2026-08-04	1000	75.8%	Testing
2026-08-02	1000	76.3%	Refactoring
2026-08-01	1000	76.0%	Testing
2026-07-31	1000	75.5%	Testing
2026-07-30	1000	76.0%	Testing
2026-07-29	1000	75.7%	Testing
2026-07-28	1000	75.2%	Refactor

Trend: Overall merge rate has held stable in the 75–77% band for the past ~30 days, with Testing consistently the top or near-top category. Today's overall rate (75.1%) is within normal range, though slightly below the 8-day average (~76%).

---

Generated by Copilot PR Prompt Analysis (Run: 31046460102)

Generated by 🔍 Copilot PR Prompt Pattern Analysis · auto · 28.8 AIC · ⌖ 6.05 AIC · ⊞ 10.9K · ◷

• expires on Aug 6, 2026, 1:04 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-06T21:04:04.294Z.

Closed by Workflow