You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Despite the HuggingFace listing showing "Weights size (bf16)", the downloaded safetensors files are actually F32. You can verify this yourself by reading the header of any shard — every tensor dtype comes back as F32.
If you're self-hosting (RunPod network volume, etc.) this doubles your storage cost unnecessarily, and if your container or pipeline doesn't pre-convert, the model loads in F32 and has to convert to BF16 so your runtime converts on the fly — wasting VRAM and GPU time on every start.
I converted them to BF16 offline using a pure numpy script (no torch required) — 4 shards, ~15 minutes, 50% size reduction, verified dtype after each shard. The converted weights load and generate identically. This saves converting them each time you spin up.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Heads up on the xl weights — they're F32 not BF16
Despite the HuggingFace listing showing "Weights size (bf16)", the downloaded safetensors files are actually F32. You can verify this yourself by reading the header of any shard — every tensor dtype comes back as F32.
If you're self-hosting (RunPod network volume, etc.) this doubles your storage cost unnecessarily, and if your container or pipeline doesn't pre-convert, the model loads in F32 and has to convert to BF16 so your runtime converts on the fly — wasting VRAM and GPU time on every start.
I converted them to BF16 offline using a pure numpy script (no torch required) — 4 shards, ~15 minutes, 50% size reduction, verified dtype after each shard. The converted weights load and generate identically. This saves converting them each time you spin up.
All reactions