Translating a video into another language is a 5-model job. yvalang streams 4 of them through one Socaity SDK. 23 languages, MIT-clean models.

Why

AI workflows are multi-model. There is no one-model-serves-all yet. Translating a video into another language needs five separate models:

1. Speech-to-text (Whisper)
2. Translation (LLM, e.g. DeepSeek)
3. Text-to-speech for synthesis
4. Voice cloning so it sounds like the original speaker
5. Lip-sync so the mouth matches the new audio

Doing this by hand is five installs, five SDKs, five auth flows, and a custom script you have to maintain. For creators that is hours of video editing, file conversions, and tool switching. For dev-creators it is the same plus glue code at scale.

yvalang streams 4 of those 5 model calls through the Socaity SDK. Lip-sync stays out of scope for this audio pilot, pair with Wav2Lip or SadTalker for the visual half. The value isn't the demo. It is what production multilingual voice cloning looks like when you stop juggling providers.

What

The audio half of the 5-model workflow, streamed through the Socaity SDK:

Model #	Stage	Backed by	Command
1	speech → text	Whisper	yvalang transcribe
2	text → text	DeepSeek V3	yvalang translate
3 + 4	text → cloned speech	Chatterbox-Multilingual (MIT)	yvalang tts
5	lip-sync (visual half)	external (Wav2Lip, SadTalker)	not in scope

The three covered primitives also chain together as yvalang batch for multi-language runs in one call. Each command is also a Python function. Pick the slice you need.

How

git clone https://github.com/SocAIty/pilot-your-voice-any-language
cd pilot-your-voice-any-language
pip install -e .
cp .env.example .env       # then fill SOCAITY_API_KEY (socaity.ai/account/secrets)

Use it as a library:

from yvalang import transcribe, translate, tts

text_es = transcribe("voice.wav")
text_fr = translate(text_es, lang="fr")
tts(text_fr, voice="voice.wav", out="hola_fr.wav", lang="fr")

Or as a CLI (each subcommand maps to one primitive plus one batch):

yvalang transcribe voice.wav
yvalang translate "Hola, encantada de conocerte." --lang fr
yvalang tts "Bonjour, ravie de te rencontrer." --voice voice.wav --lang fr --out hola_fr.wav
yvalang batch voice.wav --langs en,fr,de,ja

The batch pipeline writes one .wav per language to outputs/voice_clone_<timestamp>/, plus a manifest.json and an interactive HTML comparator so you can A/B-listen and pick the strongest takes.

If you only need one stage (just TTS, just translation), import the function you need and skip the rest. The three primitives are independent.

For direct SDK use without this wrapper, see the Socaity docs.

Use cases

• Video localization. Add the audio half of multilingual video without 5 SaaS subscriptions. Pair with a lip-sync model for the visual half.
• Voice-driven products. Ship multilingual audio in an app without writing your own model orchestration. Three imports, one SDK, 23 languages.
• Language-learning tools. Generate pronunciation samples in the learner's own voice. Less robotic feedback, better retention signal.
• Accessibility. Generate localized voiceover for tutorials, voice memos, and customer-facing audio without re-recording.

Architecture

See docs/architecture.md for the C4 container and component diagrams.

Before you publish a cloned voice

Voice cloning is powerful and easy to misuse. Things you (the person shipping the product) need to check before anything goes live:

• Get consent in writing. Only clone voices you have explicit, written permission to use. "Verbal okay" is not consent in most jurisdictions.
• Keep the watermark. The default TTS model embeds a PerTh watermark. Don't strip it. It's what lets platforms detect AI-generated audio.
• Disclose to your audience. YouTube, TikTok, Meta, and most ad platforms now require AI-generated audio to be labeled. Check the policy on your channel before publishing.
• Verify your model's license. The default is MIT and commercial-clean. If you swap to another TTS model, read its license before going commercial. Many are research-only.
• Confirm data residency for your jurisdiction. Audio runs on Socaity infrastructure in the EU by default. If you have stricter requirements (HIPAA, GDPR-sensitive workloads), validate before sending real user samples.

Languages

Chatterbox-Multilingual supports: ar, da, de, el, en, es, fi, fr, he, hi, it, ja, ko, ms, nl, no, pl, pt, ru, sv, sw, tr, zh.

Keep the spoken sentence short. Chatterbox caps each call at 300 characters and the pipeline truncates at the last sentence boundary if a translation overflows.

Layout

src/yvalang/
  transcribe.py    Whisper wrapper
  translate.py     DeepSeek V3 wrapper
  tts.py           SpeechCraft voice-clone wrapper
  batch.py         orchestrator: chains the three + manifest
  cli.py           argparse subcommands
  report.py        HTML comparator (Pillow + HTML, no model calls)
docs/architecture.md   C4 container + component
pyproject.toml         package metadata, `yvalang` console script
smoke.py               install + key presence check

---

SocAIty  ·  Docs  ·  APIs  ·  Pricing  ·  Get an API key

_{Licensed under Apache-2.0.}