tensorRT: fix verify_fp8_rope's CHECK 3, which failed on the engine it validates

[Cortexelus]

Found while validating the fp8 medium DiT for sm_120: verify_fp8_rope.py reports
CHECK3=False for the shipped fp8 engine on both architectures, so the documented
identity check for that tier has never passed.

Why it failed

The condition was len(rope_fp32) > 0 and prec_any["FP32"] > 100.

">100 FP32 layers" describes the fp32-trunk recipe. The shipped engine bakes RoPE's
cos/sin as fp32 constants, so there is no fp32 trunk — only the two runtime Fourier chains
stay fp32. That is 6 FP32 layers on sm_120 and 16 on sm_90; neither can exceed 100.

The other clause tried to prove an internal fp32 island from per-layer dtypes, but
those are dtypes at the fused blob boundary, and Myelin fuses boundaries differently per
arch. Same ONNX, same recipe, same measured fidelity:

sm_120  __myl_MaxMinDivReshMulSinCastCosCastConc      I/O ['Float','BFloat16']
sm_90   __myl_CastMaxMinDivReshMulCosSinCastCastConc  I/O ['BFloat16','BFloat16']

sm_90 absorbed the cast into the blob so its boundary reads BF16 — opposite verdicts from
a check that cannot see inside either blob. Whether the island is fp32 internally is not
decidable from this metadata, so CHECK 3 stops pretending it is. The boundary dtypes are
still printed, explicitly labelled informational.

What it checks now

RoPE must be baked: no runtime cos/sin beyond the two Fourier chains
(timestep_features, seconds_total), plus some fp32 surviving. That is arch-independent
and it catches the failure that matters — a silently failed bake means RoPE is computed at
runtime in reduced precision and drifts at long sequence, which is the bug the recipe
exists to prevent.

Fidelity belongs to a numerical test, and the check now says so: sweep velocity-cos vs the
fp32 engine across t. Calibration damage and RoPE damage both concentrate near t=1.0
and are invisible mid-schedule — a spot check at t=0.7 cannot tell a calibrated engine
from an uncalibrated one.

Also: the diagnostic whose absence cost the most time

An engine built without ProfilingVerbosity.DETAILED serializes layer names only. The
inspector returns a list of strings, every per-layer lookup finds nothing, and all three
checks report False — indistinguishable from "fp8 did not fire". build_from_onnx.py never
sets that flag, so this is the default experience for every engine in the repo.

The tool now detects it and says to rebuild with DETAILED, noting that verbosity is
metadata-only so the twin is a valid stand-in for the shipped engine. Measured on sm_120:
DETAILED twin vs shipping engine gives identical velocity-cos and speed within 0.7%.

Verification

	sm_120	sm_90
fp8 E4M3 GEMMs	176 / 181	176 / 181
fused MHA	96, all BF16	96, all BF16
cos/sin layers	2	2
CHECK 1 / 2 / 3	True / True / True	True / True / True

Still False on a names-only engine — unverifiable is not the same as verified.

🤖 Generated with Claude Code