Bazarr LocalAI Whisper Proxy

An Elysia/Bun compatibility proxy that translates Bazarr's SubGen-style Whisper requests into LocalAI's OpenAI-compatible audio API. It runs no inference model and needs no media mounts or FFmpeg.

Configure and run

cp .env.example .env
# Set LOCALAI_BASE_URL and LOCALAI_MODEL in .env
docker compose up --build -d

Configure Bazarr's Whisper ASR Docker Endpoint as http://HOST:30201. Do not point Bazarr directly at LocalAI.

For local development:

bun install
LOCALAI_BASE_URL=http://10.10.10.10:30200 LOCALAI_MODEL=whisper-1 bun run dev

API

• GET /status provides Bazarr's required version response without invoking inference.
• GET /health checks LocalAI reachability and whether LOCALAI_MODEL appears in /v1/models. It returns 503 when degraded.
• POST /asr accepts Bazarr's audio_file, splits raw 16 kHz mono PCM into bounded WAV chunks, forwards them sequentially, offsets/renumbers the returned cues, and returns one SRT.
• POST /detect-language sends only the configured total audio duration to LocalAI and returns Bazarr-compatible language JSON. For Bazarr raw PCM, it combines three equal windows sampled around 10%, 50%, and 85% of the runtime so intros and credits do not dominate detection.

The Bun HTTP idle timeout is disabled intentionally: Bazarr keeps a single request open while Whisper inference runs. Keep Bazarr's Transcription/translation timeout comfortably above the expected LocalAI processing time as well.

Upstream LocalAI calls use Bun's native fetch implementation. The proxy buffers each complete LocalAI response while LOCALAI_TRANSCRIPTION_TIMEOUT_MS remains active, so the deadline covers both inference and response-body delivery.

ASR_CHUNK_SECONDS defaults to 600 (10 minutes). Chunking prevents a feature-length raw PCM upload from becoming one opaque, failure-prone LocalAI inference. The proxy logs asr_chunk_started and asr_chunk_completed for progress. Containerized audio from future clients is still forwarded as one file because reliable container splitting would require FFmpeg.

For slower hardware, start with five-minute chunks and one transcription at a time:

ASR_CHUNK_SECONDS=300
MAX_CONCURRENT_TRANSCRIPTIONS=1

MAX_AUDIO_UPLOAD_MB is enforced both by Bun's HTTP server and by the application. Its default of 1024 MB accommodates the full raw PCM uploads Bazarr creates for feature-length media.

LocalAI enforces a separate upload limit (15 MB by default). Set LOCALAI_UPLOAD_LIMIT=1024 on the LocalAI service, not only this proxy, so LocalAI accepts the full WAV sent by /asr. If an ingress or reverse proxy sits in front of LocalAI, its body-size limit must also be raised.

Language detection prefers LocalAI's verbose_json ISO-639-1 language field and accepts top-level and common nested language/language_code shapes. Some LocalAI Whisper versions return only segments, text, and duration; for those responses, the proxy uses the lightweight local tinyld detector on the transcript. It accepts only sufficiently long, high-confidence results. If the initial distributed sample is uncertain, the proxy retries once with twice the configured duration and otherwise fails explicitly. Transcript content is never logged.

Raw audio is assumed only when encode is not true and the upload has no recognized audio MIME type. Recognized WAV, MP3, MP4, FLAC, Ogg, and WebM uploads are forwarded in their existing container.

Observed performance

One tested setup produced the following result for a one-hour TV episode (about 42 minutes of audio):

Component	Tested value
CPU	Intel Core i5-3570
GPU	AMD Radeon R5 430 2 GB
Inference server	LocalAI
Model	Whisper Base (whisper-base)
Backend	Whisper
Proxy chunking	300 seconds, sequential
Result	9/9 chunks completed; merged SRT returned with HTTP 200
Total proxy time	496.196 seconds (8 minutes 16 seconds)
Throughput	About 5.1x faster than the 42-minute audio runtime

This is a single observed run, not a guaranteed benchmark. Performance varies with LocalAI configuration, backend acceleration, media content, thermal limits, and concurrent workloads.

Translation

Translation is disabled by default because LocalAI backend support varies. When your installed LocalAI backend has been verified to expose POST /v1/audio/translations, set LOCALAI_TRANSLATION_MODEL to the working model or alias. The proxy then uses that endpoint. Without this explicit configuration, task=translate returns HTTP 501 and is never silently downgraded to transcription.

Verification

bun test
bun run typecheck
docker compose config

Real LocalAI integration still requires audio samples and access to the deployed backend. Probe /v1/models, transcription SRT, verbose JSON language detection, and translation before enabling the Bazarr provider in production.