feat(audio): native PipeWire capture backend on Linux - #1774
Conversation
|
Thank you I will take a look at this! |
|
@cjpais any chance you've had a look at this? Been running it from my own fork build for about a month now — works great, zero complaints. Happy to rebase the conflicts if you're open to merging it 👍 |
On Linux, capture is pinned to cpal's ALSA host (`get_cpal_host` -> `HostId::Alsa`), so Handy's microphone stream is an ALSA "default" client. PipeWire cannot see it as a first-class node: there is no per-application capture volume, no routing, and device enumeration only ever exposes "default" instead of the graph's real sources. Add a native PipeWire capture backend (pipewire-rs 0.10) that registers the microphone stream as a real PipeWire node, so it appears in `wpctl status` with its own settable volume and is routable per-app. It runs the PipeWire main loop on a dedicated thread (all non-Send pw objects stay on it), downmixes interleaved F32 to mono, and feeds the existing resampler -> VAD -> consumer pipeline unchanged via the shared Cmd/AudioChunk protocol. A small `Recorder` seam selects the backend: on Linux it prefers PipeWire and falls back to the cpal/ALSA path if PipeWire setup fails; non-Linux targets compile cpal-only and are unaffected. cpal stays fully compiled and available as the fallback. The MVP captures the default source (reproducing today's "default" behaviour). PipeWire-native device enumeration/selection is left as a follow-up (TODO in recorder_backend.rs, where a selected mic would map to a node.name passed through TARGET_OBJECT).
8bec23c to
0a7bc41
Compare
|
Okay, I'm gonna say I really want to merge this. I just need time to review. I'm gonna put this on my priority, so I appreciate you bumping this. This is definitely very important, and we need to merge this. I just need to check the implementation. I've probably been deferring it because it's a thousand lines of new code, but I think that this is very important. So let's definitely get this in ASAP. Hopefully, we'll have a chance to at least do some minor review this morning and rebase it |
|
Hey @olosegres I have some thoughts. I think it's probably worth doing device enumeration here. And another thing I'm wondering about is whether it makes sense on Linux to add an advanced setting for 'Audio Backend' or similar. With 3 options "Auto", "Pipewire", "ALSA". Part of the reason I'm thinking about this is that I want to make device enumeration clean, and I feel like the only way to do that is to basically have the app always know which backend is actually selected, and that auto would effectively do resolution and prefer pipewire. But if it doesn't exist, fall back. I'm curious what your thoughts are on this and if this is a set of changes that you think you're able to make. Also, I believe that we might need to add some dependencies in the CI to fix it as well if you're able to take a look at that. I do want to merge this. I'm just curious your thoughts and opinions since you're a primary Linux user and I'm not |
2 participants
Problem
On Linux, microphone capture is pinned to cpal's ALSA host (
get_cpal_host()→HostId::Alsainaudio_toolkit/audio/utils.rs). Handy's capture stream is therefore an ALSA"default"client, and PipeWire never sees it as a first-class node. On the (now near-universal) PipeWire desktop this means:"default"client. Handy therefore captures the raw, un-boosted signal and ends up dramatically quieter than every other app on the same mic (near-silent recordings, poor/empty transcriptions), with no in-app way to fix it: raising the source volume withwpctl/pavucontrol has no effect on Handy.wpctl/ pavucontrol / WirePlumber — Handy is invisible to all of that."default"on many PipeWire systems — the ALSA cpal host doesn't surface the graph's friendly per-source nodes — so the mic selector can't offer the real inputs.Change
Add a native PipeWire capture backend (
pipewire-rs0.10) that registers Handy's microphone as a real PipeWire node, so it shows up inwpctl statuswith its own settable volume and is routable per-app.pipewire_recorder.rs: runs the PipeWire main loop on a dedicated thread (all non-Sendpw objects stay on it), negotiates F32, downmixes to mono in the RTprocesscallback, and feeds the existing resampler → VAD → consumer pipeline unchanged via the sharedCmd/AudioChunkprotocol. No DSP is reimplemented.recorder_backend.rs(Recorder): a thin seam that selects the backend. On Linux it prefers PipeWire and falls back to the cpal/ALSA path if PipeWire setup fails (e.g. no running session). Non-Linux targets compile cpal-only and are behaviourally unchanged; cpal stays fully compiled as the fallback everywhere.pipewireis added only under[target.'cfg(target_os = "linux")'.dependencies](featurev0_3_44), so Windows/macOS builds are untouched.Testing
Built and run on Fedora (Asahi, aarch64) under GNOME/Wayland with PipeWire 1.6.8. Hotkey → record → transcribe works end-to-end through the native backend; the log shows
Microphone capture using native PipeWire backend, and the capture node appears inwpctl statuswith a settable volume (wpctl set-volume <id> …) — which the old ALSA client did not support (Node … does not support volume).Scope / follow-ups
This is an MVP that captures the default source (reproducing today's
"default"behaviour). PipeWire-native device enumeration/selection is intentionally deferred — there's aTODOinRecorder::open, where a user-selected mic would map to anode.namepassed throughTARGET_OBJECT(already threaded throughPipeWireRecorder::open). Happy to iterate on device selection and feature-gating in review.