tensorRT: encoder/decoder fp8 tiers + DiT precision rename (fp16mixed→fp16); retire w8_bf16 & medium bf16

[Cortexelus]

Quantized encoder + decoder tiers (SAME-S / SAME-L)

Train-free fp8 and int8-weight tiers for both encoders and decoders, each grafted onto the
published bf16 export and preserved with GPTQ + per-op precision placement. Adds a wide-profile
build script and a silence-pad node so encoders take any audio length.

The tier ONNX + sm_90 engines ship on HF (stabilityai/stable-audio-3-optimized, next to the existing
dec_dynamic_bf16.onnx / enc_dynamic_*); this PR adds the recipe, the build script, and the runtime
wiring. The bf16 baseline is the first row of each table; decoder dB is vs that baseline, encoder
dB/cos is vs eager (the latent). Speed = whole-model median @1292.

Selecting a tier: --dec-precision {canonical|fp8|w8_bf16|fp8_fast} (orthogonal to the DiT
--precision) in both CLIs (sa3_trt.py, sa3_trt_core.py); the gradio decoder picker offers them too.
Engines auto-download from HF on first use. Back-compat preserved: --dec-precision canonical still
follows --precision for the decoder (--precision fp32 → dec_dynamic_fp32.trt as before).

SAME-S decoder

file	speed	onnx	dB (music / sfx)
dec_dynamic_bf16 (baseline)	1.00×	219 MB	ref (7.25 ms @1292)
dec_w8_bf16	1.00×	59 MB	38.2 / 50.0 · transparent
dec_fp8	1.14×	58 MB	30.9 / 41.0 · near-transparent
dec_fp8_fast	1.22×	58 MB	26.1 / 36.5 · lossy

SAME-L decoder

file	speed	onnx	dB
dec_dynamic_triton_swa (baseline)	1.00×	1193 MB	ref (54.0 ms @1292)
dec_w8_bf16	1.00×	937 MB	51.4 · transparent
dec_fp8	1.15×	937 MB	43.4 · near-transparent

SAME-S encoder — dB / cos vs eager (encoder is cheap → win is size, not speed)

file	speed	onnx	dB / cos
enc_dynamic_bf16 (baseline)	1.00×	216 MB	36.4 / 0.997 (~1 ms)
enc_w8_bf16	1.00×	54 MB	36.2 / 0.997 · transparent
enc_fp8	1.08×	54 MB	29.9 / 0.986 · near-transparent
enc_fp8_fast	1.14×	54 MB	22.3 / 0.924 · lossy

SAME-L encoder — dB / cos vs eager

file	speed	onnx	dB / cos
enc_dynamic_triton_swa (baseline)	1.00×	1192 MB	46.0 / 0.9997 (~7 ms)
enc_w8_bf16	1.02×	895 MB	44.5 / 0.9995 · transparent
enc_fp8	1.03×	895 MB	30.8 / 0.989 · near-transparent

The two knobs

• fp8 = fp8 compute (tensor cores) → real speedup, ~31 dB floor from fp8 activations (GPTQ can't lift it).
• w8_bf16 = int8 weight-only storage, bf16 compute → smallest + most transparent, no speedup.

fp8 vs fp8_fast

Both run the FFN GEMMs (ff.0 / ff.2) in fp8. The difference is the attention projections:

• fp8 leaves to_qkv / to_out in bf16 compute (weight-only fp8 for storage) → attention inputs
  stay full-precision → near-transparent.
• fp8_fast runs to_qkv / to_out in fp8 too → the extra +0.08× (1.14 → 1.22×), but it
  fp8-rounds Q/K/V before the (still-bf16) attention core, and the softmax path is fp8-sensitive → the
  quality drop (31 → 26 dB). SAME-S only — SAME-L's attn projections are fp32 islands, so it has no _fast.

Sequence length

The graphs are fully dynamic — verified correct at L=1 → 8192 vs eager (odd/even/prime, no size
effect; quant tiers track bf16 identically at every length). build_tiers.py bakes a wide profile
(latent [1,1292,8192] / audio [1,·,33.5M]), so L=1..31 work (can't be chunked) and long files run
natively — the old [32,4096] was a conservative default, not a model limit. Encoders carry a
silence-pad node (pad_encoder.py) so any N is accepted; single-shot is accurate at any length (the
runtime's encode_chunked turns out to be unnecessary).

Notes for review

• Grafters reference campaign-local calibration data / checkpoints via absolute paths (see README note);
  the ready-made HF ONNX need none of that.
• SAME-L tiers are ~900 MB because attention projections stay fp32 islands — weight-only-quantizing those
  would pull them to ~500 MB (follow-up).

🤖 Generated with Claude Code