Near-duplicate comments in json output for agent

[GenosseFlosse]

OpenCodeReview Version

v1.8.6 (1b193db) windows/amd64

Operating System

Windows

Installation Method

npm (global)

LLM Provider

Other OpenAI-compatible endpoint

Bug Description

ocr review --to feature-branch --from main --format json --audience agent will output near-duplicate comments in json
e.g. see log.

Steps to Reproduce

1. run ocr on code with issues in format=json and audience=agent mode
2. Observe some duplicate comments

Expected Behavior

• unique comments, even if it costs me an llm call
• An even more problematic side effect is that inline comments in gitlab will be duplicates as well

Logs / Error Output

C:\Users\>ocr review --to feature-branch --from main --format json --audience agent
{
  "status": "complete",
  "message": "Review complete: 8 finding(s) across 9 selected item(s).",
  "summary": {
    "files_reviewed": 9,
    "comments": 8,
    "total_tokens": 409739,
    "input_tokens": 390776,
    "output_tokens": 18963,
    "elapsed": "1m16s"
  },
  "tool_calls": {
    "total": 21,
    "by_tool": {
      "code_comment": 8,
      "code_search": 7,
      "file_read": 6
    }
  },
  "comments": [
    {
      "path": "Models/asdf.m",
      "content": "The code accesses `config.asdf` before verifying that the `asdf` field exists in `config`. If `config.asdf` is missing, this will raise a runtime error before the later `assert(isfield(config, 'asdf'), ...)` can catch the problem. Add an existence check (e.g., `isfield(config, 'asdf') \u0026\u0026 isfield(config.asdf, 'skip_DB_connection')`) or move the `assert(isfield(config, 'asdf'), ...)` before the conditional.",
      "suggestion_code": "if isfield(config, 'asdf') \u0026\u0026 isfield(config.asdf, 'skip_DB_connection') \u0026\u0026 ~config.asdf.skip_DB_connection",
      "existing_code": "if ~config.asdf.skip_DB_connection",
      "start_line": 20,
      "end_line": 20,
      "category": "bug",
      "severity": "high"
    },
    {
      "path": "Models/asdf.m",
      "content": "The code accesses `config.asdf.skip_DB_connection` before confirming the `asdf` field exists, which can cause a runtime error when the field is missing. Move the existence check before the conditional or include an `isfield` guard.",
      "suggestion_code": "+if isfield(config, 'asdf') \u0026\u0026 isfield(config.asdf, 'skip_DB_connection') \u0026\u0026 ~config.asdf.skip_DB_connection",
      "existing_code": "+if ~config.asdf.skip_DB_connection",
      "start_line": 20,
      "end_line": 20,
      "category": "bug",
      "severity": "high"
    },
    {
      "path": "Models/qwertz.m",
      "content": "Using ... is clearer and safer.",
      "suggestion_code": "for k = 1:numel(blabla)",
      "existing_code": "for k = 1:nnz(blabla)",
      "start_line": 230,
      "end_line": 230,
      "category": "style",
      "severity": "low"
    }
}
...

Additional Context

No response

[lizhengfeng101]

Thanks for the detailed report and the clear repro output — this is definitely a valid bug.

We already have a DEDUP_TASK infrastructure in the scan pipeline that merges near-duplicate findings via an LLM call, and we think the fix here could either:

1. Reuse the existing dedup machinery (buildDedupCommentsJSON / applyDedupGroups) as a post-processing step in the review pipeline, or
2. Extend the REVIEW_FILTER_TASK prompt to also flag near-duplicates (same file + same line + semantically equivalent content) alongside provably-incorrect comments.

If anyone is interested in tackling this, we're happy to discuss the approach further — feel free to chime in here.

[amh1k]

Interested in tackling this issue