Enhance Final Patch Selection with Repository-Level Understanding and Majority Voting

[dcloud347]

Title: Enhance Final Patch Selection with Repository-Level Understanding and Majority Voting

Description:

The current patch selection strategy in Prometheus can be improved by incorporating repository-level semantic understanding and majority voting, to better resolve issues that span multiple files, require execution validation, or involve subtle behavioral differences between patches.

---

🔍 Proposed Design

1. Enhancing Repository-Level Understanding via Context Retrieval Agent

While our current final_patch_selector is capable of ranking and selecting patches based on textual features and model confidence, it lacks awareness of the broader repository context, which is often essential for resolving real-world issues spanning multiple files or dependent modules.

To address this limitation, we propose to use Context Retrieval Agent prior to final patch selection, with the following role:

• Goal: Augment the selector with repository-level context to improve the semantic accuracy of patch evaluation.

• Mechanism:

  • Before the selector evaluates patches, the context_retrieval_agent gathers relevant code snippets from the repository, such as:

    • Functions or classes mentioned in the issue or modified by patches
    • Upstream/downstream call sites
    • Dependencies within the module or across files

  • This context is then fed as additional input to the final_patch_selector, allowing it to make better-informed decisions.

  • Optionally, the selector can simulate lightweight dynamic execution (e.g., run a subset of existing tests) to collect behavioral traces that aid in ranking.

This enhancement retains our current selector architecture but equips it with structured semantic understanding of the codebase before making patch decisions.

2. Majority Voting on Patch Candidates

To reduce LLM hallucination risk and boost selection consistency:

• Run the selector_agent in parallel for $N$ iterations on the top-k deduplicated candidate patches.
• Each agent iteration votes for the best patch based on execution output and semantic reasoning.
• The patch with the highest number of votes is returned as final.
• Early stopping: if a patch wins the majority (i.e., > $\left\lceil \frac{N}{2} \right\rceil$) early, we can stop further evaluations to improve efficiency.

3. Integration Plan

• Improve the selector agent prompt structure
• Add voting logic with early consensus return

---

🧪 Benefits

• Improves selection accuracy for real-world multi-file or dynamic-behavior bugs
• Reduces false positives due to syntax-only comparisons
• Provides better robustness under noisy candidate patch generation