SkyRL quantization (FP8, MXFP8, NVFP4) tracker

[erictang000]

This issue tracks the development of quantization for RL using the Megatron training backend and vLLM inference backend in SkyRL.

FP8 Training

• Supporting Deepseek style blockwise (128x128) full FP8 training on Hopper + Blackwell: FP8 RL Training and Rollout Weight Synchronization #1898
• Enabling FP8 KV cache in vLLM: FP8 KV-Cache Compatibility for Hybrid vLLM Models #1899
• Enabling BF16 LoRA training with FP8 base weights
• Support MXFP8 quantization on experts: Persistent MXFP8 for experts layers in MoE #1940
• Use per-token activation scaling in FP32 (as in https://arxiv.org/pdf/2603.24477v2)
• Bump TransformerEngine version from 2.11 -> 2.16.1
• validate use of dequantized backward in TransformerEngine per (https://humansand.ai/blog/nvfp4-rl?v=3, Add NVTE_BACKWARD_OVERRIDE=high_precision|dequantized NVIDIA/TransformerEngine#2644) - use NVTE_BACKWARD_OVERRIDE=dequantized
• Improve support for specifying which layers should be quantized

INT4 QAT Training

• Support LoRA only INT4 Fake QAT training (W4A16 serving, fake INT4 quant training): [megatron] 1/n towards Kimi K2.6: fake QAT training for Qwen3.6-35B-INT4 #1862

NVFP4 Training

• Support NVFP4 Training with 4 over 6 weights/activations per https://humansand.ai/blog/nvfp4-rl?v=3