Voice gateway: full-duplex voice for any registered agent

[lezama]

What

A sidecar daemon (voice-gateway/, Python — same ship-a-daemon-in-the-plugin-repo pattern as wp-carpeta's bridge/) that gives full-duplex voice to any agent registered with the Agents API on a site running openclaWP.

browser mic (PCM 16 kHz over WSS)  ⇄  gateway.py  ⇄  Gemini Live API
                                         │ ask_agent(consulta)
                                         ▼
        POST {site}/wp-json/openclawp/v1/agenttic/{agent}
        (JSON-RPC message/send, app-password auth, sticky sessionId)

Design: one brain

Gemini Live is only the voice shell — native STT/TTS, turn-taking, barge-in. It gets exactly one tool, ask_agent, which forwards every domain request to the agenttic chat endpoint. The registered agent stays the single brain: conversational memory (sticky sessionId), destructive-tool gating (/decisions) and transcripts all stay in WordPress. The voice model is instructed to never answer domain questions from its own knowledge and to verbalize the wait while the agent thinks (tool runs async over the open audio channel).

This deliberately avoids the duplicated-brain architecture (declaring the product's full tool surface to the voice model), which would drift from the agent's prompt and bypass site-side gating.

Pieces

• gateway.py — FastAPI WS bridge browser⇄Gemini Live (BidiGenerateContent), async tool execution, per-session audio-seconds cost log, /healthz.
• static/index.html — dependency-free page: mic capture (PCM s16 16 kHz), scheduled 24 kHz playback with barge-in flush, live transcripts, text-input fallback. WS URL is computed relative to the page so a reverse-proxy path prefix works.
• README.md — setup, Caddy snippet, launchd unit, wire protocol.

Tested

• Server boots, serves the page, /healthz reports missing config, WS politely rejects unconfigured sessions.
• Live end-to-end (Gemini key + real agent) pending on the first deployment (wp-carpeta's Menta on menta).

Not in this PR

• Mic in ChatSurface (this ships a standalone page first).
• Agent-token auth (uses app passwords like the WhatsApp bridge; agents-api tokens are the natural upgrade).
• Mapping a spoken "sí" onto pending tool decisions — v1 reads the pending confirmation out loud and relays the user's reply as a normal turn.

🤖 Generated with Claude Code

[lezama]

Update: credentials now come from WordPress, shaped as a core proposal.

New POST /openclawp/v1/voice/session (includes/class-openclawp-voice-session.php):

1. Reads the Google key the canonical way — AiClient::defaultRegistry()->getProviderRequestAuthentication('google') (wp-ai-client credential store; the key never enters a REST response).
2. Mints a single-use ephemeral Live token (v1alpha/auth_tokens, model-constrained, 30 min expiry) and returns {credential, ws_url, model}.
3. Raw-key fallback exists but is behind an off-by-default filter (openclawp_voice_session_allow_api_key) — the core-shaped contract is that the stored key never leaves the server.

The gateway asks WP for its credential per session (?access_token= on the v1alpha Bidi WS, per the ephemeral-tokens docs) and keeps local GEMINI_API_KEY only as a fallback.

docs/voice-gateway-core-proposal.md is the pitch to take upstream: php-ai-client provider capability for ephemeral client credentials for realtime sessions (createEphemeralCredential('google', $constraints)), with this endpoint as the working proof of concept. Same shape covers OpenAI Realtime's client_secret flow.