[prompt-analysis] Copilot PR Prompt Analysis - 2026-08-07

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-08-07

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 739 (73.9%) | Closed: 244 (24.4%) | Open: 17 (1.7%)

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Feature Addition	226	176	44	77.9%
Testing	94	73	18	77.7%
Other	132	100	28	75.8%
Bug Fix	453	325	124	71.7%
Documentation	40	29	11	72.5%
Refactoring	55	36	19	65.5%

(Category = keyword match on title/body; rate = merged / (merged+closed), open PRs excluded from denominator)

Prompt Analysis

✅ Successful Prompt Patterns (Merged PRs)

• Average prompt length: 233 words
• 65.4% of merged PR prompts reference a specific file/path (e.g. `pkg/...` or .github/workflows/)
• Frequent non-stopword terms: workflow, run(s), actions, copilot, engine, fix
• Prompts tend to include a root-cause explanation, a concrete fix description, and a short "Tests" section

Example merged prompts:

1. PR Add PureLock: daily pure-function maximum-coverage test workflow #51107: "Adds PureLock 🔐 — a daily agentic workflow that identifies one pure Go function with weak coverage and generates a m..." → Merged
2. PR feat: merge discussion-task-miner into deep-report #51100: "Consolidates discussion-task-miner into deep-report, eliminating the redundant workflow. Both workflows mined the sa..." → Merged
3. PR Add log-parser script to cursor.md engine definition #51087: "The Cursor engine definition was missing a log-parser script, preventing the agent drain pipeline from extracting stru..." → Merged

❌ Unsuccessful Prompt Patterns (Closed PRs)

• Average prompt length: 338 words (~45% longer than merged prompts)
• Only 59.4% reference specific files — lower than merged, but not dramatically so
• Frequent terms unique to closed PRs: cve, redacted, debian, securitytracker, gpl/lgpl, license, only, high
• Cluster driven by automated dependency/license/CVE-scan PRs, which are frequently superseded, false-positive, or policy-judgment calls rather than clean code fixes

Example closed prompts:

1. PR Update container license scan policy for reviewed image findings #51025: "The daily container image scan reported license policy violations for gh-aw-firewall/cli-proxy:0.27.44, mostly from Al..." → Closed
2. PR Remediate dynamic-regexp custom-linter findings across pkg/agentdrain, stringutil, parser, cli, workflow #50995: "This daily regexpdynamicpattern custom linter flagged 23 call sites where regexp.Compile/regexp.MustCompile pattern..." → Closed
3. PR Refactor workflows to import shared Copilot sandbox standard bundle (first-wave adoption) #50970: "This PR starts extracting the repeated 'hardened Copilot agent' frontmatter bundle into a shared workflow component and..." → Closed

Key Insights

• Length matters inversely: closed-PR prompts average ~338 words vs. 233 for merged — longer prompts often signal broad/ambiguous scans (license/CVE/security-tracker sweeps) rather than a single well-defined change.
• CVE/license/security-scan prompts underperform: keywords like cve, redacted, securitytracker, gpl/lgpl dominate closed PRs — these tend to require human policy judgment or get superseded by newer scans, rather than being straightforward code fixes.
• Refactoring has the lowest success rate (65.5%) among categories — open-ended "improve/simplify" prompts appear harder for Copilot to land cleanly than scoped bug fixes or feature adds.
• File-path specificity correlates positively: merged prompts reference concrete files/paths noticeably more often (65.4% vs 59.4%), reinforcing that naming exact files/functions helps.

Recommendations

1. DO: Scope prompts to a single root cause + fix + test plan, similar to the merged bug-fix examples (concise, ~200-250 words, explicit file/function references).
2. DO: For feature/testing prompts, keep the "Root cause → Fix → Tests" structure — these categories have the highest success rates (77-78%).
3. AVOID: Broad automated-scan prompts around CVE/license/security-tracker findings without a specific, actionable code change — route these through manual triage before delegating to Copilot, or narrow scope to one finding at a time.
4. AVOID: Open-ended refactor requests ("simplify", "improve") without a concrete target — pair refactor prompts with a specific function/file and success criteria.

Historical Trends

Date	PRs	Merged	Closed	Success Rate	Top Category
2026-08-07	1000	739	244	73.9%	Bug Fix
2026-08-06	1000	752	245	75.4%	Bug Fix
2026-07-06	1000	793	193	80.4%	Testing
2026-07-05	1000	805	191	80.8%	Bug Fix
2026-07-04	1000	798	186	81.1%	Bug Fix

Trend: Overall merge success rate has declined from ~80-81% in early July to ~74-75% in early August. This coincides with a rise in closed PRs matching license/CVE-scan keywords (cve, redacted, securitytracker, gpl), suggesting an increase in automated compliance-scan-driven PRs that are harder to land compared to targeted bug fixes/features.

---

Generated by Copilot PR Prompt Analysis (Run: §31217685591

Generated by 🔍 Copilot PR Prompt Pattern Analysis · auto · 23.8 AIC · ⊞ 10.8K · ◷

• expires on Aug 8, 2026, 12:56 PM UTC-08:00

[github-actions[bot]]

Caveman say: smoke test run good. Fire burn, code build, comment land here on discussion 51190. Ug ug 🔥🦴

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• contentautofill.googleapis.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · auto · 46.3 AIC · ⌖ 2.65 AIC · ⊞ 8.7K · ◷
_{Comment /smoke-copilot to run again
Add label smoke to run again}