Prerequisite for the 4K half of #592, and for HDR at high resolution (#594).
Why
LTX-2 checkpoints ship positional_embedding_max_pos = [20, 2048, 2048] and RoPE normalizes pixel positions by that span, so an axis past 2048px is outside the trained range — it renders, but out of distribution, with no error. #668 added a guard that rejects those shapes rather than producing them silently.
Upstream reaches 4K a different way: stage 2 runs over overlapping latent tiles, each denoised at a shape the model handles well, with positions renormalized per tile so every tile looks like a sequence starting at zero. Results are recombined with a separable trapezoidal window. packages/ltx-pipelines/src/ltx_pipelines/hdr_ic_lora.py (:455-563) is the only single-GPU reference; ltx-core/modality_tiling.py and tiling.py hold the machinery.
Scope
- Tile arithmetic: interval splitting, blend window, tile enumeration. Done —
origin/tiling-primitives, 10 tests, fixtures captured by running upstream's own split_by_count in its virtualenv.
- Wire it into
render_real_two_stage_av: per-tile latent slice, per-tile positions via build_video_positions at the tile's pixel shape (which is what upstream's normalize_positions=True achieves), per-tile RNG seeded seed + tile_idx, accumulate the weighted results.
- Spatial VAE decode tiling, so the decode of a large frame does not undo the memory saving.
A bug found while porting
split_by_count silently breaks the partition of unity once the stride drops below the overlap: tiles i and i+2 then also overlap, three trapezoids stack, and the windows sum to as much as 1.16. blend has no normalization pass, so the seam is brightened rather than erroring. Measured:
| split |
window sum |
(21, 2, 8) |
1.000 |
(21, 3, 8) |
up to 1.074 |
(21, 4, 8) |
up to 1.161 |
Upstream's _clamp_tile_to_latent only prevents the ValueError, not this. Solving tile_size >= 2 * overlap gives overlap <= dim / (n + 1), which the port enforces. Upstream's shipped config is 2 tiles per axis, which can never triple-overlap, so they do not hit it — but any layout tuned for 4K would.
Verification
The property that makes tiling correct is independent of the resolution that makes it necessary, so this can be verified without 4K hardware: assert a tiled stage-2 render matches an untiled one within a documented per-pixel tolerance at 512x512x25. True 4K needs 48-80GB per upstream's own table and is not verifiable on a 24GB card.
Prerequisite for the 4K half of #592, and for HDR at high resolution (#594).
Why
LTX-2 checkpoints ship
positional_embedding_max_pos = [20, 2048, 2048]and RoPE normalizes pixel positions by that span, so an axis past 2048px is outside the trained range — it renders, but out of distribution, with no error. #668 added a guard that rejects those shapes rather than producing them silently.Upstream reaches 4K a different way: stage 2 runs over overlapping latent tiles, each denoised at a shape the model handles well, with positions renormalized per tile so every tile looks like a sequence starting at zero. Results are recombined with a separable trapezoidal window.
packages/ltx-pipelines/src/ltx_pipelines/hdr_ic_lora.py(:455-563) is the only single-GPU reference;ltx-core/modality_tiling.pyandtiling.pyhold the machinery.Scope
origin/tiling-primitives, 10 tests, fixtures captured by running upstream's ownsplit_by_countin its virtualenv.render_real_two_stage_av: per-tile latent slice, per-tile positions viabuild_video_positionsat the tile's pixel shape (which is what upstream'snormalize_positions=Trueachieves), per-tile RNG seededseed + tile_idx, accumulate the weighted results.A bug found while porting
split_by_countsilently breaks the partition of unity once the stride drops below the overlap: tiles i and i+2 then also overlap, three trapezoids stack, and the windows sum to as much as 1.16.blendhas no normalization pass, so the seam is brightened rather than erroring. Measured:(21, 2, 8)(21, 3, 8)(21, 4, 8)Upstream's
_clamp_tile_to_latentonly prevents theValueError, not this. Solvingtile_size >= 2 * overlapgivesoverlap <= dim / (n + 1), which the port enforces. Upstream's shipped config is 2 tiles per axis, which can never triple-overlap, so they do not hit it — but any layout tuned for 4K would.Verification
The property that makes tiling correct is independent of the resolution that makes it necessary, so this can be verified without 4K hardware: assert a tiled stage-2 render matches an untiled one within a documented per-pixel tolerance at 512x512x25. True 4K needs 48-80GB per upstream's own table and is not verifiable on a 24GB card.