Skip to content

fix: opus codec issue#318

Open
0xrushi wants to merge 5 commits intoSimpleOpenSoftware:devfrom
0xrushi:fix/omi-transcription-from-dev-clean
Open

fix: opus codec issue#318
0xrushi wants to merge 5 commits intoSimpleOpenSoftware:devfrom
0xrushi:fix/omi-transcription-from-dev-clean

Conversation

@0xrushi
Copy link
Contributor

@0xrushi 0xrushi commented Feb 26, 2026

Summary by CodeRabbit

  • New Features

    • Added support for advanced authentication in audio streaming connections.
    • Improved codec parameter handling for WebSocket connections, with automatic normalization and configuration.
  • Bug Fixes

    • Enhanced audio payload processing to properly detect and handle audio header configurations, improving streaming reliability.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 26, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR enhances WebSocket URL building with codec normalization and authentication parameters, while introducing Opus audio header-stripping detection throughout the audio processing pipeline. A new utility function identifies stripped headers based on metadata, and this flag is propagated from client initialization through packet processing on the backend.

Changes

Cohort / File(s) Summary
Frontend WebSocket URL Configuration
app/src/hooks/useAudioStreamingOrchestrator.ts
Updated URL builders normalize base URLs, enforce ws/wss protocol and /ws path, normalize codec parameter (opus for standard, pcm for phone), and conditionally append authentication query parameters when advanced mode is active.
Backend Opus Header Detection
backends/advanced/src/advanced_omi_backend/utils/omi_codec_utils.py, backends/advanced/tests/test_omi_codec_utils.py
New utility module adds is_opus_header_stripped() function to detect BLE header stripping on Opus payloads. Function handles optional audio metadata, converts string representations to booleans, and defaults to True when metadata is absent. Comprehensive unit tests cover boolean flags, string variations, and missing data scenarios.
Backend Audio Processing Integration
backends/advanced/src/advanced_omi_backend/controllers/websocket_controller.py
Integrates header-stripping detection into WebSocket audio handling. Computes audio_start_data on audio initialization and detects stripped headers using the new utility. Propagates strip_header boolean parameter through audio chunk processing, replacing partial function application with direct decoder calls.
Wearable Client Payload
extras/local-wearable-client/backend_sender.py
Adds "opus_header_stripped": True flag to initial audio_start WebSocket payload to signal header stripping status to the backend. Minor logging formatting adjustment included.

Sequence Diagram

sequenceDiagram
    participant Client as Client
    participant Backend as WebSocket Controller
    participant Detector as Header Detector
    participant Decoder as Audio Decoder
    participant Producer as Audio Stream Producer

    Client->>Backend: audio_start payload with opus_header_stripped=True
    Backend->>Detector: is_opus_header_stripped(audio_start_data)
    Detector-->>Backend: strip_header=True
    Backend->>Backend: Initialize session with strip_header flag
    
    Client->>Backend: audio_chunk (Opus payload)
    Backend->>Decoder: decode_packet(opus_payload, strip_header=True)
    Decoder->>Decoder: Conditionally strip BLE header
    Decoder-->>Producer: Decoded audio data
    Producer->>Producer: Process audio stream
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 70.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title 'fix: opus codec issue' is vague and does not clearly summarize the actual changes made in the pull request. Consider revising the title to be more specific, e.g., 'fix: handle Opus codec header stripping in WebSocket connections' or similar, to better convey the actual changes.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

0xrushi and others added 3 commits February 26, 2026 15:24
- Updated `useAudioStreamingOrchestrator` to ensure WebSocket URLs are correctly formatted and include codec parameters for Opus and PCM.
- Introduced a new utility function `is_opus_header_stripped` to determine if incoming Opus payloads have the BLE header removed, improving audio processing accuracy.
- Modified the WebSocket handling in `websocket_controller.py` to accommodate the new header stripping logic and updated audio session initialization.
- Added unit tests for the new utility function to ensure correct behavior across various input scenarios.
- Improved URL normalization for WebSocket connections by trimming user input and converting HTTP(S) schemes to WS(S).
- Ensured the presence of the `/ws` endpoint path in the WebSocket URL.
- Updated codec handling to enforce Opus as the stream codec, either by replacing existing codec parameters or appending it if missing.
@AnkushMalaker
Copy link
Collaborator

@coderabbitai review

@AnkushMalaker AnkushMalaker self-assigned this Feb 27, 2026
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 27, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@app/src/hooks/useAudioStreamingOrchestrator.ts`:
- Around line 66-68: The code declares `const isAdvanced` twice in
`useAudioStreamingOrchestrator.ts`, which causes a compilation error; remove the
duplicate declaration so there is only a single `const isAdvanced =
settings.jwtToken && settings.isAuthenticated;` (keep the one used by the
surrounding logic) and update any subsequent references to `isAdvanced` if
necessary to use that single variable.

In
`@backends/advanced/src/advanced_omi_backend/controllers/websocket_controller.py`:
- Line 1410: The fallback for checking whether to strip the Opus header is
inconsistent: change the getattr call that reads getattr(client_state,
"opus_header_stripped", False) to use True so it matches the default behavior of
is_opus_header_stripped(); i.e., replace the default False with True so both the
property check and is_opus_header_stripped() assume the header is already
stripped and avoid incorrect stripping when an audio chunk arrives before
audio-start.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting

📥 Commits

Reviewing files that changed from the base of the PR and between 75f0d70 and 0f281e9.

📒 Files selected for processing (5)
  • app/src/hooks/useAudioStreamingOrchestrator.ts
  • backends/advanced/src/advanced_omi_backend/controllers/websocket_controller.py
  • backends/advanced/src/advanced_omi_backend/utils/omi_codec_utils.py
  • backends/advanced/tests/test_omi_codec_utils.py
  • extras/local-wearable-client/backend_sender.py

Comment on lines +66 to 68
const isAdvanced = settings.jwtToken && settings.isAuthenticated;

const isAdvanced = settings.jwtToken && settings.isAuthenticated;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Duplicate variable declaration will cause compilation error.

Lines 66 and 68 both declare const isAdvanced. This is a syntax error in TypeScript/JavaScript — you cannot redeclare a const variable in the same scope.

🐛 Proposed fix: Remove the duplicate declaration
     if (/[?&]codec=/i.test(url)) {
       url = url.replace(/([?&])codec=[^&]*/i, '$1codec=opus');
     } else {
       const sep = url.includes('?') ? '&' : '?';
       url = url + sep + 'codec=opus';
     }
-    const isAdvanced = settings.jwtToken && settings.isAuthenticated;
-
     const isAdvanced = settings.jwtToken && settings.isAuthenticated;
     if (isAdvanced) {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const isAdvanced = settings.jwtToken && settings.isAuthenticated;
const isAdvanced = settings.jwtToken && settings.isAuthenticated;
if (/[?&]codec=/i.test(url)) {
url = url.replace(/([?&])codec=[^&]*/i, '$1codec=opus');
} else {
const sep = url.includes('?') ? '&' : '?';
url = url + sep + 'codec=opus';
}
const isAdvanced = settings.jwtToken && settings.isAuthenticated;
if (isAdvanced) {
🧰 Tools
🪛 Biome (2.4.4)

[error] 68-68: Shouldn't redeclare 'isAdvanced'. Consider to delete it or rename it.

(lint/suspicious/noRedeclare)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@app/src/hooks/useAudioStreamingOrchestrator.ts` around lines 66 - 68, The
code declares `const isAdvanced` twice in `useAudioStreamingOrchestrator.ts`,
which causes a compilation error; remove the duplicate declaration so there is
only a single `const isAdvanced = settings.jwtToken &&
settings.isAuthenticated;` (keep the one used by the surrounding logic) and
update any subsequent references to `isAdvanced` if necessary to use that single
variable.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks legit? @0xrushi

audio_stream_producer,
payload,
_decode_packet,
not getattr(client_state, "opus_header_stripped", False),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Default value inconsistency with is_opus_header_stripped.

The fallback here is False (assume header is NOT stripped → strip it), while is_opus_header_stripped() defaults to True (assume header IS stripped → don't strip). If an audio chunk somehow arrives before audio-start, this mismatch could lead to incorrect decoding.

While the protocol should guarantee audio-start precedes audio-chunk, aligning the defaults would be safer:

🛡️ Proposed fix to align defaults
                 await _handle_omi_audio_chunk(
                     client_state,
                     audio_stream_producer,
                     payload,
                     _decode_packet,
-                    not getattr(client_state, "opus_header_stripped", False),
+                    not getattr(client_state, "opus_header_stripped", True),
                     user.user_id,
                     client_id,
                     packet_count,
                 )
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
not getattr(client_state, "opus_header_stripped", False),
not getattr(client_state, "opus_header_stripped", True),
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@backends/advanced/src/advanced_omi_backend/controllers/websocket_controller.py`
at line 1410, The fallback for checking whether to strip the Opus header is
inconsistent: change the getattr call that reads getattr(client_state,
"opus_header_stripped", False) to use True so it matches the default behavior of
is_opus_header_stripped(); i.e., replace the default False with True so both the
property check and is_opus_header_stripped() assume the header is already
stripped and avoid incorrect stripping when an audio chunk arrives before
audio-start.

Copy link
Collaborator

@AnkushMalaker AnkushMalaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one duplicate that coderabbit caught,
the other, I think I wanna try my devkit once and see if we can remove this header thing across

Comment on lines +66 to 68
const isAdvanced = settings.jwtToken && settings.isAuthenticated;

const isAdvanced = settings.jwtToken && settings.isAuthenticated;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks legit? @0xrushi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants