Skip to content

Discord/Slack bot citation footer number mismatches inline citation marker when a document is cited under two different raw numbers #14334

Description

@NielsJorna

Summary

When the same underlying document is retrieved and cited more than once under two different raw citation numbers within one answer, the inline [[N]] marker in the answer text can show a different number than the one listed in the bot's "Sources" footer for that same document/link — even though both point to the correct URL.

Root cause

In onyx/chat/citation_processor.py, DynamicCitationProcessor._process_citation():

# Track cited documents and create CitationInfo only for new citations
if doc_id not in self.cited_document_ids:
    self.cited_document_ids.add(doc_id)
    self.cited_documents_in_order.append(search_doc)
    citation_info_list.append(
        CitationInfo(
            citation_number=num,
            document_id=doc_id,
        )
    )

CitationInfo (and therefore anything downstream built from response.citation_info, e.g. a "Sources" footer) is only emitted the first time a document_id is cited. If the same document appears in the retrieved/ranked context at two different positions (so the model cites it under two different raw numbers, e.g. [4] and later [8]), the inline text still renders both [[4]](url) and [[8]](url) correctly (the link is baked directly into the text at each occurrence), but only the first occurrence's number ends up in citation_info. Anything built from citation_info — such as the Discord bot's source-list footer in onyx/onyxbot/discord/handle_message.py::_append_citations() — then shows that document under its first number only, which no longer matches the second inline marker.

Repro

Ask a scoped persona a question where the same doc/page is likely to surface via two different chunks/search hits (broad "what does X offer" style questions over a vendor doc set tend to trigger this). Example that reproduced it:

  • Persona: a topic-scoped assistant over vendor documentation (Relevant Digital docs in this case), accessed via the Discord bot integration.
  • Question: "What does Relevant Digital's platform offer for SSP management?"
  • Result: the answer body contains a citation marker [8] linking to docs.relevant-digital.com/.../mapping-ad-server-and-ssp-placement-ids-..., but the rendered "Sources" footer lists that same URL as source 4. Clicking [8] inline resolves to the correct URL — the link itself isn't broken, only the visible number is inconsistent with the footer.

Impact

Cosmetic/trust issue rather than a functional break — every citation link still resolves to the correct source. But a citation number in the body that doesn't match its own footer entry is confusing and undermines confidence in the citations more broadly, especially in a low-context surface like Discord where there's no separate "Cited Sources" panel to cross-reference against (unlike the web UI).

Suggested fix direction

When collapsing/deduping citations by document_id for citation_info, either:

  1. Rewrite all occurrences of the raw number(s) for a given document_id in the output text to match the single deduped number used in citation_info (similar to what onyx/chat/citation_utils.py::collapse_citations() already does for a different renumbering case), or
  2. Track and expose all raw numbers that map to a given document_id, so a footer can be built that matches every inline occurrence.

Environment

  • Onyx version: v4.3.0 (self-hosted, plain docker compose deployment)
  • Surface: Discord bot integration (onyx/onyxbot/discord/)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions