feat(zoom): index Cloud Recording transcripts from configured meetings by Subash-Mohan · Pull Request #14482 · onyx-dot-app/onyx

Subash-Mohan · 2026-09-04T13:24:17Z

Description

An admin can list Zoom meeting ids, and every recorded occurrence of those meetings becomes a searchable document
Registers Zoom as a document source so the connector is reachable from indexing
Indexes every occurrence of a recurring meeting, not just the most recent one
Skips meetings that were never cloud-recorded, and re-checks ones whose transcript was still processing (ZOOM_TRANSCRIPT_LAG_BUFFER_HOURS, default 72)
Admin UI to configure the connector is not included — it lands with webinar support, so this is reachable from tests but not yet from the product

How Has This Been Tested?

Unit tests for discovery, processing, checkpointing and session types (60 tests); daily suite scaffolded against the real Zoom API.

Additional Options

[Optional] Please cherry-pick this PR to the latest release version.
[Optional] Override Linear Check

Summary by cubic

Adds a Zoom connector that indexes Cloud Recording transcripts from configured meeting IDs, making each recorded occurrence of those meetings a searchable document. The admin UI to configure it isn't included yet, so the connector is reachable from tests but not the product.

New Features

Registers Zoom as a document source so indexing and Slack answers pick it up.
Indexes every occurrence of a recurring meeting, not just the most recent one.
Skips meetings that were never cloud-recorded and retries transcripts still processing (ZOOM_TRANSCRIPT_LAG_BUFFER_HOURS, default 72).
Requires at least one meeting ID; validation rejects an empty config.

Bug Fixes

Rate limits, timeouts, expired credentials, and broken connections now fail the run so the checkpoint is preserved and the next run resumes on the same item; errors scoped to one transcript or session record a failure carrying the exception and move on.
Transcripts Zoom marks as not downloadable are skipped before the download attempt; still-processing ones are retried on a later sync.

^{Written for commit 0c602f8. Summary will update on new commits.}

github-actions · 2026-09-04T13:27:59Z

Full-stack Preview (frontend + backend)

Status	Preview	Commit	Updated
✅	https://0c602f8-onyx.preview.onyxcorp.dev/	`0c602f8`	2026-09-05 07:04:16 UTC

greptile-apps · 2026-09-04T13:43:44Z

Greptile Summary

The PR adds checkpointed discovery and indexing of Zoom Cloud Recording transcripts for configured meeting IDs.

Registers Zoom as a document source and connector.
Expands recurring meetings into occurrence-specific documents.
Adds bounded discovery, transcript readiness checks, lag-window polling, and retry-aware failure handling.
Adds connector configuration and comprehensive unit and daily-test coverage.

Confidence Score: 5/5

The PR appears safe to merge with no outstanding actionable defects identified.

The latest changes correctly consolidate transcript readiness checks, preserve diagnostic exceptions without serializing them, and retain checkpoint position for transient HTTP 408 failures. The three previous threads were manually resolved without explanatory replies and therefore do not remain outstanding.

Important Files Changed

Filename	Overview
backend/onyx/connectors/zoom/connector.py	Coordinates checkpointed Zoom discovery and occurrence processing while preserving progress across invocations.
backend/onyx/connectors/zoom/recordings/discovery.py	Expands configured meeting IDs into bounded, poll-window-filtered occurrence work.
backend/onyx/connectors/zoom/recordings/processing.py	Applies transcript readiness checks, downloads and parses VTT content, and emits Zoom documents or actionable failures.
backend/onyx/connectors/zoom/recordings/models.py	Defines checkpoint work models and classifies transient request failures that must preserve checkpoint position.
backend/tests/unit/onyx/connectors/zoom/test_zoom_processing.py	Covers restricted transcripts, optional download flags, attached exceptions, and systemic HTTP failures.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Configured Zoom meeting IDs] --> B[List meeting occurrences]
    B --> C{Occurrence in poll window?}
    C -- No --> B
    C -- Yes --> D[Store occurrence in checkpoint work queue]
    D --> E[Fetch transcript metadata]
    E --> F{Downloadable?}
    F -- Not ready or unavailable --> G[Skip and revisit through overlap window]
    F -- Yes --> H[Download and parse VTT]
    H --> I[Create occurrence-specific Zoom document]
    I --> J[Index searchable transcript]

_{Reviews (3): Last reviewed commit: "fix(zoom): gate the transcript download ..." | Re-trigger Greptile}

cubic-dev-ai

All reported issues were addressed across 18 files

_{Reply with feedback, questions, or to request a fix.

Re-trigger cubic}

Adds the connector itself on top of the client: an admin lists Zoom meeting ids, and each recorded occurrence of those meetings becomes a searchable document. Registers DocumentSource.ZOOM so the connector is reachable from the indexing pipeline; the admin UI to configure it comes with webinar support. A meeting id is expanded into its past occurrences rather than fetched directly, because a recurring meeting records each run separately and the bare id only reaches the most recent one. Structure follows the two axes that vary. Discovery mechanisms are resumable sources that emit occurrences and own an opaque cursor, so a crash resumes instead of restarting, and host/group discovery can page Zoom's own way later. Meeting-versus-webinar differences sit behind a session-type handler. connector.py holds only checkpoint mechanics. Two limits worth knowing. Zoom's occurrence listing takes no date or page parameters, so a long-running meeting is paged client-side and each page after the first re-lists it; without that cap the checkpoint is rewritten in full on every invocation. And an occurrence is selected by when the meeting ran, while its transcript appears later, so each poll reaches back ZOOM_TRANSCRIPT_LAG_BUFFER_HOURS (default 72) to catch ones that were still processing. Zoom publishes no maximum for that lag. Document ids carry the session type because a targeted reindex is handed ids and nothing else, and changing the scheme after indexing orphans what is already stored. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBNe54dXPELTXXGdk1uYR6

A ConnectorFailure ends the attempt COMPLETED_WITH_ERRORS, which Onyx counts as successful, so the next run rebuilds the checkpoint over a newer poll window. Rate limits, expired credentials and network errors were all landing there, which permanently dropped any occurrence older than ZOOM_TRANSCRIPT_LAG_BUFFER_HOURS. Targeted reindex cannot recover those either, since it is keyed on document ids and a discovery failure only produces an entity failure. Errors that will hit every remaining occurrence now propagate, so the attempt fails and the next run resumes from its checkpoint on the same work item. Errors specific to one transcript or session still record a ConnectorFailure and move on. Waiting and retrying is left to the client, which already mounts a urllib3 Retry honouring Zoom's Retry-After. Exhausting it raises RetryError rather than HTTPError, so a sustained 429 is classified on the exception type, not on a status code. Also carry the session type in the discovery EntityFailure, since 111 is a legal id for both a meeting and a webinar. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBNe54dXPELTXXGdk1uYR6

The classifier listed the transport errors it knew about, so a body that stopped mid-transfer raised JSONDecodeError or ChunkedEncodingError and was read as one bad session. Discovery then skipped that meeting, and the next run's poll window no longer reached its history. HTTPError is the only requests error where a response came back to judge, so classify on that and treat every other RequestException as systemic. This also covers whatever requests adds later. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBNe54dXPELTXXGdk1uYR6

Occurrence timestamps were pinned to January 2026 while the poll window ends at time.time(). Discovery drops anything outside that window, so on a machine whose clock predates the pinned date every result set empties and the count assertions fail for a reason unrelated to what they test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBNe54dXPELTXXGdk1uYR6

Subash-Mohan · 2026-09-05T06:48:51Z

@greptile @cubic

cubic-dev-ai · 2026-09-05T06:49:00Z

@greptile @cubic

@Subash-Mohan I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 18 files

_{Reply with feedback, questions, or to request a fix.

Re-trigger cubic}

ZoomTranscript.is_downloadable existed but nothing called it, so the download ran whenever a url was present. Zoom returns a url alongside can_download=False or a restriction reason, and each of those cost a download that Zoom then refused. NOT_READY still logs as a retry rather than a warning, since it is the one restriction that clears on its own. Document failures now carry the exception. The runner gates its Sentry report and its threshold re-raise on that field, so both Zoom download failures were invisible there while discovery's were not. Classify HTTP 408 as systemic. The client's Retry covers 429 but not 408, so a timed-out request was reaching the per-occurrence path and skipping that occurrence for good. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBNe54dXPELTXXGdk1uYR6

Subash-Mohan · 2026-09-05T07:01:52Z

@greptile @cubic

cubic-dev-ai · 2026-09-05T07:02:01Z

@greptile @cubic

@Subash-Mohan I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 18 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

Subash-Mohan mentioned this pull request Sep 4, 2026

feat(zoom): add Zoom API client and transcript parsing #14481

Open

2 tasks

Subash-Mohan force-pushed the Subash-Mohan/zoom-connector branch from 501c955 to 67e22ca Compare September 4, 2026 13:39

Subash-Mohan marked this pull request as ready for review September 4, 2026 13:40

Subash-Mohan requested a review from a team as a code owner September 4, 2026 13:40

greptile-apps Bot reviewed Sep 4, 2026

View reviewed changes

Comment thread backend/onyx/connectors/zoom/connector.py

Comment thread backend/onyx/connectors/zoom/recordings/discovery.py

cubic-dev-ai Bot reviewed Sep 4, 2026

View reviewed changes

Comment thread backend/onyx/connectors/zoom/recordings/discovery.py

Comment thread backend/onyx/connectors/zoom/recordings/processing.py

Comment thread backend/tests/unit/onyx/connectors/zoom/test_zoom_connector.py

Subash-Mohan and others added 4 commits September 5, 2026 11:15

Subash-Mohan force-pushed the Subash-Mohan/zoom-connector branch from 67e22ca to d29865f Compare September 5, 2026 06:47

greptile-apps Bot reviewed Sep 5, 2026

View reviewed changes

Comment thread backend/onyx/connectors/zoom/recordings/processing.py Outdated

cubic-dev-ai Bot reviewed Sep 5, 2026

View reviewed changes

Comment thread backend/onyx/connectors/zoom/recordings/processing.py Outdated

Comment thread backend/onyx/connectors/zoom/recordings/processing.py

Comment thread backend/onyx/connectors/zoom/recordings/models.py Outdated

cubic-dev-ai Bot reviewed Sep 5, 2026

View reviewed changes

Subash-Mohan mentioned this pull request Sep 5, 2026

feat(zoom): index webinars alongside meetings #14516

Open

2 tasks

Uh oh!

Conversation

Subash-Mohan commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How Has This Been Tested?

Additional Options

Summary by cubic

Uh oh!

github-actions Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Subash-Mohan commented Sep 5, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 5, 2026

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Subash-Mohan commented Sep 5, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 5, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Subash-Mohan commented Sep 4, 2026 •

edited

Loading

github-actions Bot commented Sep 4, 2026 •

edited

Loading

greptile-apps Bot commented Sep 4, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading