Skip to content

feat: use official deepgram-sdk for voice agent - #10

Open
GregHolmes wants to merge 2 commits into
mainfrom
feat/use-deepgram-sdk
Open

feat: use official deepgram-sdk for voice agent#10
GregHolmes wants to merge 2 commits into
mainfrom
feat/use-deepgram-sdk

Conversation

@GregHolmes

Copy link
Copy Markdown

What

Migrate the Deepgram-facing side of this Voice Agent proxy from a raw websocket-client connection to the official deepgram-sdk (client.agent.v1).

How

  • Replaced websocket.create_connection(wss://agent.deepgram.com/...) with deepgram.agent.v1.connect(), running the SDK's blocking start_listening() in a background thread and forwarding agent messages to the browser (binary audio as-is, JSON events via model_dump_json).
  • Kept the transparent-proxy contract: the browser still drives the full agent protocol. Microphone audio -> connection.send_media(...); browser JSON control frames (Settings, Update*, InjectAgentMessage, KeepAlive, ...) are forwarded verbatim, so no frontend changes are required.
  • requirements.txt: dropped websocket-client and the now-unused requests, added deepgram-sdk>=7.6.0. deepgram.toml: sdk = "deepgram-sdk".
  • Flask routes, browser WebSocket, JWT auth, /api/session, /api/metadata, monkeypatch, and shutdown unchanged.

Notes

  • The old code logged the Deepgram dg-request-id from the raw upgrade response headers; the SDK connection does not expose those, so that log line was dropped (minor observability change).
  • test_app.py is pre-existing dead code unrelated to this change: it imports socketio / dg_connection from a removed flask-socketio implementation and does not match the current flask-sock app. Left untouched; it should be rewritten separately.

Verified

  • Python 3.12 venv, pip install -r requirements.txt (resolved deepgram-sdk 7.6.0), python -c "import app" imports cleanly.

Manual verification needed

  • Full conversation round-trip (Settings applied, mic audio, agent text + audio) against a real API key.
  • Confirm verbatim JSON passthrough works for every browser control frame.
  • Replace the stale test_app.py.

@GregHolmes GregHolmes self-assigned this Jul 30, 2026
…SDK major

- On EventType.ERROR, forward a structured {type: Error, code: PROVIDER_ERROR}
  frame to the browser before teardown, restoring the pre-migration contract
  (previously a mid-session Deepgram error only closed the socket with no
  payload).
- Pin deepgram-sdk>=7.6.0,<8 so a future breaking major can't be pulled in.
- Document why control frames go through the private _send(): agent.v1 offers
  only typed senders and no public raw/dict send, so a transparent proxy has
  no public path (tracking a public raw send upstream).
@GregHolmes
GregHolmes marked this pull request as ready for review July 31, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant