[feat] lab module#2
Merged
Merged
Conversation
- Register wenetspeech_drama in datasets package - Implement adapter that filters segments by subsets containing 'D' (drama) - Add asr-drama-wenetspeech.toml suite with whisper-large-v3 baseline - Cover the adapter with 10 unit tests under tests/lab/ - Update README to document the new dataset and suite
…gnment - Add /lab route with four tabs: Datasets, Experiments, Leaderboard, Regression - Implement labApi client with auto fallback to mock data on network errors - Provide rich mock dataset (6 scenarios, 8 suites, 6 datasets, 7 runs) so the page works without the lab server (toggle via VITE_LAB_USE_MOCK=1) - Show a SourceBadge (live / mock) so users always know what they look at - Reuse shared PageContainer and StatusBadge, align colors, table skeleton, pill tabs and empty/error states with TaskListPage UI conventions - Add bilingual (zh/en) i18n entries for the whole module - Register lab in App router and main navigation
Inventory current capabilities (datasets, scenarios, suites/runs, leaderboard, regression, mock layer) and turn them into a structured requirement document with clickable code references, gap analysis and a P0/P1/P2 roadmap to guide further iteration.
Two new GT-anchored evaluation paths, both offline-tested with fabricated data (no downloads), filling the two biggest gaps from the dataset-landscape research. tts-clone — the dubbing/TTS stage had no GT metric, only e2e-dub's intrinsic score: - metrics/speaker.py: SIM = cosine over ECAPA embeddings (reuses translip's in-tree speaker_embedding; lazy + injectable + degrades to sim=None, never crashes) - scenarios/tts_clone.py: clone reference voice -> re-transcribe -> SIM (primary) + CER - tts_synth.py: lab-side worker reusing translip.dubbing qwen voice-clone core only (avoids the server adapters' heavy eager-import), isolated subprocess - datasets/synthetic_clone.py + folder .clone.txt/.ref.wav sidecars + suite - GroundTruth gains clone_text / clone_ref_wav magicdata-ramc — extends CER+DER from meetings into spontaneous dialogue: - datasets/magicdata_ramc.py: dedicated parser for RAMC's "[start,end] speaker gender,dialect text" transcripts (NOT TextGrid), reusing the shared RTTM/SRT emitters; tolerant of flat or WAV/TXT layouts - suites/asr-diar-ramc.toml (collar 0.25 / 2 speakers -> official CER 19.1/DER 7.96) Tests: +19 (tests/lab now 104 passing). Design doc at docs/translip-lab-tts-clone-eval.zh-CN.md. Format + real-TTS paths flagged as "verify against real data/models" in describe()/docs (cannot download here). translip core untouched - lab stays one-way-dependent and deletable.
The lab server lists scenarios/datasets/suites dynamically, so the new tts-clone scenario, synthetic-clone / magicdata-ramc datasets, and their suites appear in live mode automatically. This wires up the rest: - ScenarioTag: render the tts-clone tag (音色克隆 / Voice Clone) and normalize hyphenated backend names (e2e-dub, tts-clone) to the underscore label keys, so live data renders localized tags too (not just mock) - leaderboard/regression: include `sim` (higher-is-better) in the primary-metric picker so tts-clone runs surface their SIM score - experiments: infer dataset/tags for the tts-clone-synthetic + asr-diar-ramc suites - labMock.ts: add the new scenario/datasets/suites + two demo runs so the offline demo represents the new capabilities - i18n: tagClone (zh/en) Verified in-browser (system Chrome via Playwright): mock and live modes both render the new datasets/suites/tag and the tts-clone run's SIM metric; live shows "实时数据" against the real :8799 server with 0 console errors.
… footgun) The cache was keyed only by config + input fingerprints, so changing a scenario's invoke/score logic (or a metric it uses) without touching config/inputs silently served stale cached scores. Add Scenario.version (default 1) into the cache key; bump it to invalidate. Also recorded in each run's scenario_meta for traceability. --no-cache still bypasses. Tests: scenario_cache_key version sensitivity + default-is-1; runner-level version-bump-invalidates-cache (tests/lab 107 passing).
…PRD P0-1/2/4)
Leaderboard rows now link to a per-run detail view the lab server already had data
for (GET /api/lab/runs/{id}) but the UI never surfaced.
- RunDetailPage: run header, meta cards (samples/scenarios/started/elapsed),
aggregates table (primary metric / mean / micro / std / scored-failed-skipped),
and a per-sample results table. Reads the real run-manifest shape with defensive
fallbacks to the mock summary shape.
- P0-2 failure traceability: failed samples show an expandable stderr/traceback panel.
- P0-4 report download: GET /api/lab/runs/{id}/report.md (run_to_markdown) + a
download button (live fetch; mock builds the markdown client-side).
- labMock: enriched run details (full-shape aggregates + per-sample results incl. a
failure) so the offline demo represents the page; i18n detail block (zh/en).
Tests: backend report endpoint (tests/lab 108 passing); frontend build + 189 vitest
+ eslint clean. Browser-verified (system Chrome) in mock AND live (fabricated real
manifest) modes: detail renders, error expands, 0 console errors live.
…UI polling (PRD P0-3)
POST /api/lab/runs was fire-and-forget; it now submits a tracked job with a real
lifecycle (queued -> running -> succeeded/failed), serialized through a single worker
so heavy ML runs don't stampede a 16 GB host, with captured logs. Job id == run id,
so a finished job links straight to its detail page.
- server/jobs.py: JobManager (queue + single worker, log capture, in-memory + JSON
records under <runs>/.jobs/ that survive a restart; orphaned running/queued jobs
are marked failed on reload).
- server/app.py: lazy per-runs-dir JobManager; POST /runs submits a job;
GET /api/lab/jobs and GET /api/lab/jobs/{id} (+ log tail).
- frontend: triggerRun returns job_id; ExperimentsTab shows a polling JobStatus that
resolves to a "view results" link on success / "run failed"; mock job state machine
for the offline demo; i18n (zh/en).
- PRD 6.1: P0-1..P0-4 marked done.
Tests: JobManager lifecycle incl. serial execution + log capture + orphan recovery,
plus endpoints (tests/lab 115 passing). Browser-verified: mock job flow (run ->
status -> results link -> detail), and a live server actually running the worker to
a tracked terminal state with the subprocess output captured.
The leaderboard mixed all scenarios, making single-scenario decisions hard. Add a filter row above the table: - scenario filter — when set, runs are filtered to that scenario AND ranked/shown by that scenario's metric (handles scenario and scenario@arm aggregate keys), not just the first metric found across the run - status filter (finished/running/failed) + a run-id/suite search box, with a "<shown> / <total>" count Pure frontend over existing run data; i18n (zh/en). Time-window filter deferred. Verified: build + 189 vitest + eslint clean; browser (system Chrome) — scenario, status, and search filters each narrow to the right rows (0 page errors).
MasamiYui
added a commit
that referenced
this pull request
Jun 29, 2026
…ith sidechain ducking
Two coupled features landed together because they touch the same UI surface
(NewTaskPage commentary settings) and share the same render pipeline.
== Viral-recap narrator presets (5 new voices) ==
* src/translip/commentary/voices.py: add 5 narrator-recap-* presets
(classic / suspense / hype / gossip / sichuan-funny) — all routed through
Qwen3-TTS CustomVoice using the existing 9 named speakers, no new model.
* tests/test_commentary_voices.py: extend coverage to assert 14-preset total
and unique-id invariants.
* frontend/src/pages/NewTaskPage.tsx → NarratorVoicePicker: split into two
accessible <section role="region"> groups — "Built-in narrator voices" and
"🔥 Viral short-video recap" (with NEW badge). The "borrow from source"
escape hatch stays outside both groups.
* frontend/src/pages/__tests__/NarratorVoicePicker.grouping.test.tsx: new
vitest covering zh/en section labels, NEW badge, group membership and the
empty-viral-list fallback.
== Built-in BGM presets (6 mood beds, learning-only) ==
Backend:
* src/translip/commentary/bgm.py: new registry of 6 BgmPreset dataclasses
(suspense / hype / warm / documentary / comedy / action) with gain_db,
duck_db, license metadata and resolve_bgm_path() three-state semantics
(None | preset id | arbitrary path).
* src/translip/server/atomic_tools/schemas.py: CommentaryRenderToolRequest
gains bgm_preset / bgm_file_id / bgm_gain_db / bgm_duck_db with a
validator forcing duck_db <= 0 (it's an attenuation).
* src/translip/commentary/render.py: when BGM is supplied on an OST=0 clip,
build_clip_command now emits a 3-input filtergraph —
[bgm_raw][narr] sidechaincompress threshold=0.05 ratio=duck attack=150
release=600 -> [bgm]
[src][bgm][narr] amix=inputs=3, alimiter=limit=0.97
Input #2 uses -stream_loop -1 so a short placeholder loops under any
clip length. Without BGM the legacy 2-input mix is preserved
byte-for-byte (backwards compatible).
* src/translip/server/atomic_tools/adapters/commentary_render.py: resolve
bgm_file_id (upload) > bgm_preset > None and forward 3 params to render.
* src/translip/server/routes/config.py: GET /api/config/bgm-presets and
/bgm-presets/{id}/preview (FileResponse audio/wav).
Assets (CC0, algorithmically synthesised):
* scripts/build_bgm_placeholders.py: pure-Python (wave + math.sin) WAV
builder — drone/heartbeat/pad/arp/comedy/action moods, 30s mono 16-bit
44.1 kHz. Re-runnable, no external deps.
* assets/bgm/*.wav: 6 generated placeholders (~2.6 MB each). Verified via
ffprobe + an end-to-end /preview round-trip.
* assets/bgm/LICENSE.md: CC0 declaration + replacement table pointing at
Pixabay/Uppbeat/Bensound/FMA + compliance disclaimer ("for local learning
and research only; replace with licensed tracks before publishing").
Frontend:
* frontend/src/api/config.ts: new BgmPresetInfo, bgmPresetPreviewUrl(),
configApi.bgmPresets(). Both narratorVoices() and bgmPresets() now
defensively coerce non-array payloads to [] so a bad-cached or
HTML-fallback response can't crash callers.
* frontend/src/types/index.ts: TaskConfig gains commentary_bgm_preset /
commentary_bgm_gain_db / commentary_bgm_duck_db.
* frontend/src/pages/NewTaskPage.tsx: new BgmPresetPicker component
(violet-themed section, NEW badge, mood chip, "No BGM" opt-out at the
top, independent fetch/abort/play state machine). Wired into the
commentary settings Field below the narrator picker.
* frontend/src/pages/__tests__/BgmPresetPicker.test.tsx: 8 vitest cases —
zh/en section labels, NEW badge, opt-out outside the section, click
emits id, aria-pressed on selection, empty registry, and a regression
guard against non-array payloads (the items.map crash reported in
the IDE console).
== Verification ==
* pytest tests/test_commentary_bgm.py
tests/test_commentary_voices.py
tests/test_atomic_tools_adapters/test_commentary_render.py
-> 48 passed
* vitest BgmPresetPicker.test.tsx + NarratorVoicePicker.grouping.test.tsx
-> 13 passed
* GetDiagnostics -> [] (0 errors / 0 warnings)
* curl /api/config/bgm-presets -> 6 entries
curl /api/config/bgm-presets/bgm-suspense-dark/preview
-> HTTP 200 audio/wav 2.6 MB, RIFF 16-bit mono 44.1 kHz
License note: bundled WAVs are CC0 algorithmic synthesis suitable for
verifying the audio chain only. Replace assets/bgm/*.wav in-place with
licensed tracks (same filename) for any published work — no code change
required.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.