-
Notifications
You must be signed in to change notification settings - Fork 3
iOS Stereo Playout/Recording #58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: patch/m137.5
Are you sure you want to change the base?
iOS Stereo Playout/Recording #58
Conversation
Made the voice-processing bypass flag mutable so the new setter can update it without tripping the compiler (sdk/objc/native/src/audio/voice_processing_audio_unit.h:88, sdk/objc/native/src/audio/voice_processing_audio_unit.h:147). Annotated SetupAudioBuffersForActiveAudioSession, CreateAudioUnit, and the bypass helper to run on the ADM thread and added runtime RTC_DCHECK_RUN_ON guards, clearing the thread-safety diagnostics (sdk/objc/native/src/audio/audio_device_ios.h:201, sdk/objc/native/src/audio/audio_device_ios.h:204, sdk/objc/native/src/audio/audio_device_ios.h:212, sdk/objc/native/src/audio/audio_device_ios.mm:848, sdk/objc/native/src/audio/audio_device_ios.mm:909).
Stereo toggles now track per-direction channel counts, push the updated preferences into the shared audio-session config, and drive the bypass helper based on what callers request. sdk/objc/native/src/audio/audio_device_ios.mm:155 caches desired output/input channels from RTCAudioSessionConfiguration, pushes them back through UpdateAudioSessionChannelPreferences() (sdk/objc/native/src/audio/audio_device_ios.mm:841), and bases bypass decisions on the max of requested and active counts (sdk/objc/native/src/audio/audio_device_ios.mm:813). sdk/objc/native/src/audio/audio_device_ios.mm:905 reads the actual AVAudioSession channel counts when the session comes up, logs any fallback, and resets AudioParameters so the buffer wiring follows hardware limits. sdk/objc/native/src/audio/audio_device_ios.mm:1313 and sdk/objc/native/src/audio/audio_device_ios.mm:1370 implement stereo availability/toggle paths that update desired counts, refresh the audio device buffer, and recreate the fine buffer when needed. sdk/objc/native/src/audio/audio_device_module_ios.mm:340 lets ADM callers flip stereo modes directly, deferring buffer updates and bypass decisions to the backend instead of short-circuiting.
Stereo Channel Plumbing Tracked desired playout/record channel counts up front and wired them into the audio-session prefs plus bypass logic (sdk/objc/native/src/audio/audio_device_ios.mm:172, sdk/objc/native/src/audio/audio_device_ios.h:214). Added ApplyChannelConfigurationChange() to tear down/reconfigure/restart the VPIO when toggles flip, with session locking and restart handling (sdk/objc/native/src/audio/audio_device_ios.mm:856). VoiceProcessingAudioUnit now stores per-direction channel counts, reprograms stream formats, and exposes a setter the ADM can call before (re)initializing (sdk/objc/native/src/audio/voice_processing_audio_unit.h:94, sdk/objc/native/src/audio/voice_processing_audio_unit.mm:227, sdk/objc/native/src/audio/voice_processing_audio_unit.mm:576). Recording/playout callbacks size buffers by frames * channels, update silence checks, and stream interleaved PCM to FineAudioBuffer so stereo pipes stay aligned (sdk/objc/native/src/audio/audio_device_ios.mm:428, sdk/objc/native/src/audio/audio_device_ios.mm:492). Stereo setters now validate route capabilities, refresh FineAudioBuffer, call the VPIO channel setter, and roll back cleanly if reconfiguration fails (sdk/objc/native/src/audio/audio_device_ios.mm:1395, sdk/objc/native/src/audio/audio_device_ios.mm:1484). SetupAudioBuffers re-reads hardware channel counts after AVAudioSession activation and pushes them to the VPIO so we adapt if iOS falls back to mono (sdk/objc/native/src/audio/audio_device_ios.mm:980).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR enables stereo audio playout and recording capabilities for iOS audio devices by implementing channel configuration management and automatic voice processing bypass when multi-channel audio is requested.
Key changes include:
- Addition of channel configuration methods and state management for stereo audio
- Automatic voice processing bypass when stereo is enabled (since iOS voice processing only supports mono)
- Dynamic audio session reconfiguration to apply channel changes during active streaming
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
File | Description |
---|---|
voice_processing_audio_unit.mm | Adds channel configuration support and updates audio format handling for multi-channel streams |
voice_processing_audio_unit.h | Adds new API methods for channel configuration and voice processing bypass control |
audio_device_module_ios.mm | Updates stereo configuration methods to delegate to the underlying audio device implementation |
audio_device_ios.mm | Implements comprehensive stereo support with session reconfiguration and voice processing management |
audio_device_ios.h | Adds method declarations and member variables for stereo audio support |
audio_device_module.h | Updates documentation to clarify voice processing bypass behavior with stereo capture |
Comments suppressed due to low confidence (1)
sdk/objc/native/src/audio/voice_processing_audio_unit.mm:349
- Missing return statement after the closing brace. The function should return a value after the error handling block.
}
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
sdk/objc/native/src/audio/audio_device_ios.mm:1
- Missing return statement after the error case. The function continues execution even when SetStereoRecording fails, which may not be the intended behavior.
/*
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
sdk/objc/native/src/audio/audio_device_ios.mm:593
- Missing return statement before the closing brace. The function should return 0 to match the expected return type.
}
}
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
sdk/objc/native/src/audio/voice_processing_audio_unit.mm:351
- Missing return statement. The function should return the result of the SetStereoRecording call, but it always returns 0 regardless of success or failure.
}
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
sdk/objc/native/src/audio/voice_processing_audio_unit.mm:1
- Missing return statement after line 346. The function will continue execution even after reporting an error, which could lead to unexpected behavior.
/*
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.
Comments suppressed due to low confidence (1)
sdk/objc/native/src/audio/voice_processing_audio_unit.mm:351
- Missing return statement. The function should return 0 on success after the closing brace on line 347.
}
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Stereo support is exposed via RTCAudioSessionConfiguration and the existing WebRTC audio device module; no extra glue code is required.
Enable stereo up front
Call this once before the WebRTC audio session is activated (e.g. right after building your RTCPeerConnectionFactory). The native ADM will request multi-channel hardware, rebuild the voice-processing I/O unit with the right stream description, and automatically bypass the hardware voice-processing stage when stereo capture is enabled.
Toggle stereo at runtime
Route changes
When iOS reports a new audio route, WebRTC re-reads the hardware channel counts. If the route only exposes one channel (e.g. Bluetooth SCO), the ADM falls back to mono automatically, updates its buffers, and continues streaming. The “desired” stereo setting remains cached, so returning to a stereo-capable route re-enables multi-channel I/O without additional app logic.
Manual Validation (iOS device)
Built-in route (speaker + mic)
configureForStereoIfAvailable()
invoked before audio start.StereoPlayout
/StereoRecording
reporttrue
and the call audio is healthy.Plug in a stereo-capable headset (wired or AirPlay)
Switch to a mono-only route (e.g., Bluetooth SCO headset)
Stereo*IsAvailable
returnsfalse
and the call still works.Return to a stereo-capable route