Skip to content

Add wave for the recording mic + unify web cameras #1329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shomix wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Add wave for the recording mic + unify web cameras #1329

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9969977
add mic wave
Jun 19, 2026
f7f8eb2
unify web camera
Jun 19, 2026
58c9443
design wave
Jun 19, 2026
fcc2e41
merge main
Jun 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 5 additions & 102 deletions templates/clips/desktop/src/app.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1051,20 +1051,6 @@ export function App() {
setCameraOn(false);
}),
);
// The bubble relays its device lists (it holds the grant in the
// local-camera path). Used by the picker when our own enumeration is empty.
const trackRelay = (
event: string,
set: (devices: MediaDeviceInfo[]) => void,
) =>
track(
listen<{ devices?: Array<{ deviceId: string; label: string }> }>(
event,
(ev) => set((ev.payload?.devices ?? []) as MediaDeviceInfo[]),
),
);
trackRelay("clips:camera-devices", setBubbleCameras);
trackRelay("clips:mic-devices", setBubbleMics);
// Query the CURRENT visibility on mount in case the event already
// fired before React subscribed.
getCurrentWindow()
Expand Down Expand Up @@ -1138,8 +1124,6 @@ export function App() {
const wantsCamera = mode !== "screen" && cameraOn;
const nativeFullscreenRecordingActive =
mode !== "camera" && shouldUseNativeFullscreenRecording(source);
const bubbleUsesLocalCamera =
nativeFullscreenRecordingActive && localRecordingMode !== "separate";
// Ref mirror of `isRecording || recordingFlowActive` so cleanup (which
// captures the dep-snapshot value) can still see the CURRENT flow state
// at the moment it actually runs. Without this, if `recordingFlowActive`
Expand Down Expand Up @@ -1215,85 +1199,6 @@ export function App() {
"[clips-popover] bubble session start — acquiring camera + showing bubble",
);

if (bubbleUsesLocalCamera) {
const localStartTimers: Array<ReturnType<typeof setTimeout>> = [];
let localReadyUnlisten: (() => void) | null = null;
const emitLocalCameraStart = (reason: string) => {
if (cancelled) return;
console.log(
"[clips-popover] starting local bubble camera — %s",
reason,
);
emit("clips:bubble-start-local-camera", {
cameraId: cameraId || null,
}).catch((err) => {
if (!cancelled) {
console.warn(
"[clips-popover] emit local bubble camera start failed:",
err,
);
}
});
};

listen("clips:bubble-ready", () => emitLocalCameraStart("ready"))
.then((u) => {
if (cancelled) {
try {
u();
} catch {
// ignore
}
} else {
localReadyUnlisten = u;
}
})
.catch(() => {});

invoke("show_bubble")
.then(() => {
// The bubble's React listener may mount just after the Rust window
// reports as shown. Send a few idempotent starts; the bubble ignores
// repeats for the same camera but this avoids a first-show race.
for (const delay of [100, 500, 1000]) {
localStartTimers.push(
setTimeout(
() => emitLocalCameraStart(`show-bubble+${delay}ms`),
delay,
),
);
}
})
.catch((err) => {
if (!cancelled) {
console.error("[clips-popover] local bubble start failed:", err);
setCameraError(`Camera unavailable: ${err?.message ?? err}`);
}
});

return () => {
cancelled = true;
for (const timer of localStartTimers) clearTimeout(timer);
try {
localReadyUnlisten?.();
} catch {
// ignore
}
emit("clips:bubble-stop-local-camera", {}).catch(() => {});
const recordingInFlight = recordingFlowGateRef.current;
console.log(
"[clips-popover] local bubble session end — recordingInFlight=%o stillActive=%o",
recordingInFlight,
bubbleActiveRef.current,
);
// Skip teardown when the bubble is still wanted (only the capture
// source changed). Hiding here would race the re-run's show_bubble.
if (!recordingInFlight && !bubbleActiveRef.current) {
invoke("hide_overlays").catch(() => {});
}
};
}

navigator.mediaDevices
.getUserMedia({
video: {
Expand Down Expand Up @@ -1411,14 +1316,13 @@ export function App() {
// the bubble correctly). Hiding here mid-flow would kill the
// on-screen bubble window the user sees during the recording.
// Also skip when the bubble is still wanted and only the capture
// source changed (window ↔ full-screen flips `bubbleUsesLocalCamera`,
// re-running this effect): hiding would race the re-run's show_bubble
// and close the window out from under it.
// source changed (e.g. cameraId flip re-runs this effect): hiding
// would race the re-run's show_bubble and close the window out from under it.
if (!recordingInFlight && !bubbleActiveRef.current) {
invoke("hide_overlays").catch(() => {});
}
};
}, [bubbleActive, bubbleUsesLocalCamera, cameraId]);
}, [bubbleActive, cameraId]);

// ---- auto-size popover to content --------------------------------------
// The Tauri window is fixed-size via tauri.conf.json, but our content
Expand Down Expand Up @@ -1685,9 +1589,7 @@ export function App() {
// effect's deps still include `isRecording`, so the stream + bubble
// + pump stay alive for the entire recording.
const preAcquiredCameraStream =
mode !== "screen" && cameraOn && !bubbleUsesLocalCamera
? bubbleStreamRef.current
: null;
mode !== "screen" && cameraOn ? bubbleStreamRef.current : null;
// Flip the ownership flag BEFORE kicking off the recorder. Any
// bubble-session cleanup that fires after this point must leave the
// tracks alone — the recorder now owns them. Cleared in the stop /
Expand Down Expand Up @@ -2171,6 +2073,7 @@ export function App() {
onToggle={setMicOn}
systemAudio={systemAudioOn}
onSystemAudioToggle={setSystemAudioOn}
meterActive={popoverVisible && !isRecording}
Comment thread
shomix marked this conversation as resolved.
Comment thread
shomix marked this conversation as resolved.
/>
</div>

Expand Down
32 changes: 26 additions & 6 deletions templates/clips/desktop/src/components/MediaDeviceRow.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { useMemo } from "react";
import { CameraIcon, CheckIcon, ChevronDown, MicIcon } from "./Icons";
import { Switch } from "./Switch";
import { useRowMenu } from "./useRowMenu";
import { useMicMeter } from "../hooks/useMicMeter";

function Toggle({
on,
Expand All @@ -25,13 +26,26 @@ function Toggle({
);
}

function MicWave() {
// Live mic level meter — a single wave line driven by real audio. The hook
// owns the analyser and writes the path's `d`; the line oscillates around the
// center and flattens when silent.
function MicWave({ deviceId, active }: { deviceId: string; active: boolean }) {
const pathRef = useMicMeter({ deviceId, active });

return (
<span className="mic-wave" aria-hidden>
<span className="bar b1" />
<span className="bar b2" />
<span className="bar b3" />
<span className="bar b4" />
<svg
className="mic-wave-svg"
viewBox="0 0 100 24"
preserveAspectRatio="none"
>
<path
ref={pathRef}
className="mic-wave-path"
d="M 0 12 L 100 12"
fill="none"
/>
</svg>
</span>
);
}
Expand All @@ -47,6 +61,7 @@ export function MediaDeviceRow({
onToggle,
systemAudio,
onSystemAudioToggle,
meterActive = true,
}: {
kind: "camera" | "mic";
devices: MediaDeviceInfo[];
Expand All @@ -58,6 +73,7 @@ export function MediaDeviceRow({
onToggle: (v: boolean) => void;
systemAudio?: boolean;
onSystemAudioToggle?: (v: boolean) => void;
meterActive?: boolean;
}) {
const current = useMemo(
() =>
Expand Down Expand Up @@ -108,6 +124,11 @@ export function MediaDeviceRow({
title={label}
>
<span className="row-label">{label}</span>
{kind === "mic" && on ? (
<MicWave deviceId={selectedId} active={on && meterActive} />
) : (
<span className="row-flex" aria-hidden />
)}
<span className="row-chev" aria-hidden>
<ChevronDown />
</span>
Expand All @@ -117,7 +138,6 @@ export function MediaDeviceRow({
onChange={onToggle}
label={kind === "camera" ? "Camera" : "Microphone"}
/>
{kind === "mic" && on ? <MicWave /> : null}
{open ? (
<div className="row-menu" role="menu">
<button
Expand Down
54 changes: 4 additions & 50 deletions templates/clips/desktop/src/hooks/useMediaDevices.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,6 @@ function concreteMediaDeviceId(value: string | null | undefined): string {
return id && !isPseudoMediaDeviceId(id) ? id : "";
}

// Prefer this page's own enumeration; fall back to the bubble-relayed list
// when the popover can't see labels itself (local-camera path).
function preferEnumerated(
own: MediaDeviceInfo[],
relayed: MediaDeviceInfo[],
): MediaDeviceInfo[] {
return own.length > 0 ? own : relayed;
}

function isSelectableMediaDevice(device: MediaDeviceInfo): boolean {
return !!concreteMediaDeviceId(device.deviceId);
}
Expand Down Expand Up @@ -77,10 +68,6 @@ export interface MediaDevicesState {
selectedMicLabel: string;
cameraDevices: MediaDeviceInfo[];
micDevices: MediaDeviceInfo[];
// Device lists relayed from the bubble page when it owns the camera grant;
// the popover effect feeds these in from `clips:camera-devices` / mic events.
setBubbleCameras: (devices: MediaDeviceInfo[]) => void;
setBubbleMics: (devices: MediaDeviceInfo[]) => void;
loadDevices: () => Promise<void>;
requestDeviceAccess: (kind: "camera" | "mic") => Promise<void>;
}
Expand All @@ -97,8 +84,6 @@ export function useMediaDevices({
// (full-screen / local-camera path). The popover page can't enumerate device
// labels itself there without muting the live bubble, so we use these lists
// when our own enumeration comes back empty.
const [bubbleCameras, setBubbleCameras] = useState<MediaDeviceInfo[]>([]);
const [bubbleMics, setBubbleMics] = useState<MediaDeviceInfo[]>([]);
const [cameraId, setCameraId] = useState<string>(() =>
loadString(CAM_KEY, ""),
);
Expand All @@ -114,14 +99,8 @@ export function useMediaDevices({
);

const selectedMicId = useMemo(() => concreteMediaDeviceId(micId), [micId]);
const cameraDevices = useMemo(
() => preferEnumerated(cameras, bubbleCameras),
[cameras, bubbleCameras],
);
const micDevices = useMemo(
() => preferEnumerated(mics, bubbleMics),
[mics, bubbleMics],
);
const cameraDevices = cameras;
const micDevices = mics;
const selectedMicLabel = useMemo(
() =>
selectedMicId
Expand Down Expand Up @@ -175,10 +154,7 @@ export function useMediaDevices({
// popover. If the user has picked a concrete mic, use that exact device;
// otherwise leave labels locked until a real user action needs access.
try {
// While the bubble owns the camera, even this audio-only probe trips
// WebKit's single-page capture-exclusion and blacks the bubble. Don't
// probe — leave mic labels locked until the bubble is gone.
if (bubbleActiveRef.current || !selectedMicId) {
if (!selectedMicId) {
await loadDevices();
return;
}
Expand All @@ -191,34 +167,14 @@ export function useMediaDevices({
// permission denied — labels stay empty until the user grants
}
await loadDevices();
}, [bubbleActiveRef, loadDevices, selectedMicId]);
}, [loadDevices, selectedMicId]);

const requestDeviceAccess = useCallback(
async (kind: "camera" | "mic") => {
try {
if (!navigator.mediaDevices?.getUserMedia) {
throw new Error("Device selection is not available in this WebView.");
}
// When the camera bubble is live, ANY getUserMedia in this page —
// camera OR mic — hits WebKit's single-page capture-exclusion (one
// process, one webview): the bubble page's camera stream gets muted
// and goes black with no reliable unmute. The exclusion is not
// per-kind, so an audio-only probe blacks the bubble just the same.
// So we never probe here while the bubble owns the camera. Instead the
// bubble page — the only page that can capture without muting its own
// camera — does the probe and relays the device list back to us.
if (bubbleActiveRef.current) {
if (kind === "mic") {
// Ask the bubble to probe + relay the mic list (it has no mic
// grant of its own, so it must open a transient mic stream).
emit("clips:refresh-mics", {
micId: selectedMicId || null,
}).catch(() => {});
}
// Cameras are already relayed when the bubble's local camera starts.
await loadDevices();
return;
}
const stream = await navigator.mediaDevices.getUserMedia(
kind === "camera"
? { video: true, audio: false }
Expand Down Expand Up @@ -299,8 +255,6 @@ export function useMediaDevices({
selectedMicLabel,
cameraDevices,
micDevices,
setBubbleCameras,
setBubbleMics,
loadDevices,
requestDeviceAccess,
};
Expand Down
Loading
Loading