Skip to content

feat: integrate Sarvam AI transcription and dual-channel system audio capture - #104

Open
magy070 wants to merge 3 commits into
vicharanashala:mainfrom
magy070:sarvam-ai
Open

feat: integrate Sarvam AI transcription and dual-channel system audio capture#104
magy070 wants to merge 3 commits into
vicharanashala:mainfrom
magy070:sarvam-ai

Conversation

@magy070

@magy070 magy070 commented Aug 3, 2026

Copy link
Copy Markdown

Summary

This PR adds support for Sarvam AI Transcription (enabling highly accurate regional Indian language transcription) and implements a dual-channel audio setup on the teacher dashboard to capture both local microphone and desktop system audio (like Zoom/YouTube/Google Meet tab audio).

Key Features Added

  1. Sarvam AI Integration:
    • Added backend service to route transcription requests to the Sarvam AI API.
    • Implemented an automatic backend fallback: if the local Whisper transcription server is unreachable, the system seamlessly routes requests to Sarvam AI.
  2. Dual-Channel System Audio Capture:
    • Implemented browser-side tab audio capture using Electron's desktop capturer and browser display media API.
    • Created an audio mixing service that combines local mic input and system audio into a single stream for transcription.
  3. Audio Setup & Onboarding Wizard:
    • Added a First-Time Audio Setup Wizard to guide teachers in testing and selecting their primary and secondary input devices.
    • Built a System Audio Setup Modal providing instructions on using virtual audio cables on unsupported browsers (Firefox/Safari).

Files Created/Modified

  • Backend:
    • backend/src/services/sarvamTranscriptionService.js (New - API wrapper)
    • backend/src/routes/transcription.js (Added provider-specific routes and fallback checks)
    • backend/src/config.js & backend/.env.example (Added Sarvam configuration variables)
  • Frontend:
    • frontend/src/services/audioCaptureService.js (New - Dual audio capture logic)
    • frontend/src/components/AudioSetupWizard.jsx (New - Onboarding UI)
    • frontend/src/components/MicDeviceSelector.jsx (New - Device selector dropdowns)
    • frontend/src/components/SystemAudioSetupModal.jsx (New - Setup instruction guide)
    • frontend/src/pages/RoomDetailPage.jsx (Integrated wizard and recording controllers)
    • frontend/src/components/RoomSettingsModal.jsx & frontend/src/services/serverTranscriptionService.js (Integrated settings)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant