[WIP] Sudhu/megatron ifu core 0.16.0 sync by sudhu2k · Pull Request #119 · ROCm/Megatron-LM

sudhu2k · 2026-03-03T01:25:55Z

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

…ice correctly for CUDA context creation / hypothetical memory allocations (NVIDIA#2710) Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

…NVIDIA#2723) Signed-off-by: John St John <jstjohn@nvidia.com> Signed-off-by: John St. John <jstjohn@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

…#2852) Signed-off-by: John St. John <jstjohn@nvidia.com>

Signed-off-by: Robin Zhang <robinz@nvidia.com>

Co-authored-by: root <root@gpu-h100-0348.cm.cluster> Co-authored-by: root <root@gpu-h100-0193.cm.cluster> Co-authored-by: root <root@gpu-h100-0082.cm.cluster> Co-authored-by: root <root@gpu-h100-0495.cm.cluster> Co-authored-by: William Dykas <wdykas@cw-pdx-cs-001-vscode-02.cm.cluster> Co-authored-by: root <root@gpu-h100-0213.cm.cluster> Co-authored-by: root <root@gpu-h100-0435.cm.cluster> Co-authored-by: root <root@gpu-h100-0188.cm.cluster> Co-authored-by: root <root@gpu-h100-0032.cm.cluster> Co-authored-by: root <root@gpu-h100-0023.cm.cluster> Co-authored-by: root <root@gpu-h100-0368.cm.cluster> Co-authored-by: root <root@gpu-h100-0203.cm.cluster> Co-authored-by: root <root@gpu-h100-0229.cm.cluster> Co-authored-by: root <root@gpu-h100-0123.cm.cluster> Co-authored-by: root <root@gpu-h100-0217.cm.cluster> Co-authored-by: root <root@gpu-h100-0496.cm.cluster> Co-authored-by: root <root@gpu-h100-0022.cm.cluster> Co-authored-by: root <root@gpu-h100-0176.cm.cluster> Co-authored-by: root <root@gpu-h100-0190.cm.cluster>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Robin Zhang <robinz@nvidia.com>

…VIDIA#2752)

…NVIDIA#2751) Co-authored-by: Philip Petrakian <pgpetrak@gmail.com>

…VIDIA#2738)" (NVIDIA#2884) Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: kunlunl <kunlunl@nvidia.com> Signed-off-by: jianbinc <shjwudp@gmail.com> Co-authored-by: jianbinc <shjwudp@gmail.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

…A#2794) Co-authored-by: Xin Yao <xiny@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

…g `--decoder-first-pipeline-num-layers` & `--decoder-last-pipeline-num-layers` (NVIDIA#2947)

Signed-off-by: Maanu Grover <maanug@nvidia.com>

Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com> Co-authored-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Co-authored-by: root <root@gpu-h100-0348.cm.cluster> Co-authored-by: root <root@gpu-h100-0193.cm.cluster> Co-authored-by: root <root@gpu-h100-0082.cm.cluster> Co-authored-by: root <root@gpu-h100-0495.cm.cluster> Co-authored-by: William Dykas <wdykas@cw-pdx-cs-001-vscode-02.cm.cluster> Co-authored-by: root <root@gpu-h100-0213.cm.cluster> Co-authored-by: root <root@gpu-h100-0435.cm.cluster> Co-authored-by: root <root@gpu-h100-0188.cm.cluster> Co-authored-by: root <root@gpu-h100-0032.cm.cluster> Co-authored-by: root <root@gpu-h100-0023.cm.cluster> Co-authored-by: root <root@gpu-h100-0368.cm.cluster> Co-authored-by: root <root@gpu-h100-0203.cm.cluster> Co-authored-by: root <root@gpu-h100-0229.cm.cluster> Co-authored-by: root <root@gpu-h100-0123.cm.cluster> Co-authored-by: root <root@gpu-h100-0217.cm.cluster> Co-authored-by: root <root@gpu-h100-0496.cm.cluster> Co-authored-by: root <root@gpu-h100-0261.cm.cluster>

…ernorm. (NVIDIA#2434) Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com>

Signed-off-by: Hongbin Liu <hongbinl@nvidia.com>

Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com> Co-authored-by: Youngeun Kwon <youngeunk@nvidia.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

This reverts commit ffbc43f.

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com>

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Co-authored-by: Rabeeh Mahabadi <rkarimimahab@nb-hel-cs-001-vscode-02.cm.cluster> Co-authored-by: Sanjeev Satheesh <sasatheesh@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

Signed-off-by: Santosh Bhavani <santosh.bhavani@live.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

…offload` (NVIDIA#2874)

Phlip79 and others added 30 commits January 6, 2026 21:51

[docs] Cleanup homepage (NVIDIA#2823)

5af715b

[docs] Update oncall doc (NVIDIA#2822)

8a59fb5

Make default for rerun_mode=disabled not terminate with non-fatal rer… (

c2327e7

NVIDIA#2773)

Bugfix: ensure spawned persistent checkpoint worker sets its CUDA dev…

5950971

…ice correctly for CUDA context creation / hypothetical memory allocations (NVIDIA#2710) Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

Implementation of a more flexible optimizer/scheduler override system (…

41cedd6

…NVIDIA#2723) Signed-off-by: John St John <jstjohn@nvidia.com> Signed-off-by: John St. John <jstjohn@nvidia.com>

chore: rotate oncall schedule

64bc482

ci(fix): PyPI upload (NVIDIA#2843)

144049d

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci(fix): Don't fail on empty var (NVIDIA#2850)

ed5bc5c

Signed-off-by: oliver könig <okoenig@nvidia.com>

Add RL support for MOEs (NVIDIA#2742)

c64f227

ci(fix): GH release version tag (NVIDIA#2854)

ae1a2f1

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci(hotfix): Disable flaky test

7e5e16b

Signed-off-by: oliver könig <okoenig@nvidia.com>

Reduce the scope of the side stream around DDP initialization (NVIDIA…

4ba9f46

…#2852) Signed-off-by: John St. John <jstjohn@nvidia.com>

Manually update first oncall rotation (NVIDIA#2855)

49dfee2

Remove flaky iteration time functional test (NVIDIA#2862)

5fa42ec

Signed-off-by: Robin Zhang <robinz@nvidia.com>

build: Bump jet-client (NVIDIA#2876)

8d6c604

Signed-off-by: oliver könig <okoenig@nvidia.com>

Change oncall team name (NVIDIA#2861)

8b10a64

Dynamic Inference | Evict and re-compute context requests. (NVIDIA#2738)

c8ac1fe

Fix CUDA RNG Tracker (NVIDIA#2641)

965cfd3

Signed-off-by: Robin Zhang <robinz@nvidia.com>

[main] feat(moe): Support attention output gate for Qwen3-Next (3/4) (N…

65dccab

…VIDIA#2752)

[main] feat(moe): Support moe shared expert gate for Qwen3-Next (2/4) (…

bddd0a8

…NVIDIA#2751) Co-authored-by: Philip Petrakian <pgpetrak@gmail.com>

[docs] Fix docs and add generation doc (NVIDIA#2882)

0b5be24

Revert "Dynamic Inference | Evict and re-compute context requests. (N…

43b4471

…VIDIA#2738)" (NVIDIA#2884) Signed-off-by: Charlie Truong <chtruong@nvidia.com>

FP8 params support for megatron-fsdp (MXFP8/Blockwise) (NVIDIA#2239)

8f2f700

Signed-off-by: kunlunl <kunlunl@nvidia.com> Signed-off-by: jianbinc <shjwudp@gmail.com> Co-authored-by: jianbinc <shjwudp@gmail.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

docs: fix broken images, links, and typos across documentation (NVIDI…

ed29157

…A#2794) Co-authored-by: Xin Yao <xiny@nvidia.com>

ci(fix): Release version (NVIDIA#2873)

424a26d

Signed-off-by: oliver könig <okoenig@nvidia.com>

Assign mcore-oncall instead of user (NVIDIA#2879)

371ee52

tests: Disable Mamba MOE model test after 43b4471 (NVIDIA#2886)

74db1ce

Signed-off-by: oliver könig <okoenig@nvidia.com>

Fix mamba moe unit test after commit reversion (NVIDIA#2888)

7fe7f48

Improve error messages in mamba moe unit test (NVIDIA#2889)

ed461d6

Phlip79 and others added 30 commits January 29, 2026 21:33

Fix coderabbit instructions error (NVIDIA#3150)

f58b6d6

Force input ids generated by mock dataset are < vocab_size (NVIDIA#2945)

063624b

Add a check to make sure we are distributing all the layers when usin…

4652e7b

…g `--decoder-first-pipeline-num-layers` & `--decoder-last-pipeline-num-layers` (NVIDIA#2947)

Update copy-pr-bot.yaml [skip ci]

18deeff

Automatically choose available ports in ZMQ (NVIDIA#2278)

67f3515

Generate arguments from TransformerConfig (NVIDIA#2896)

639c08a

Signed-off-by: Maanu Grover <maanug@nvidia.com>

Fix for PR-2142 (NVIDIA#3165)

4cd9563

Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com> Co-authored-by: oliver könig <okoenig@nvidia.com>

ci: Onboard more GB200 tests (NVIDIA#3145)

6de6362

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci(hotfix): Alert for GB200 (NVIDIA#3168)

de15117

Signed-off-by: oliver könig <okoenig@nvidia.com>

Fix SFTDataset truncation bug (NVIDIA#3158)

7952d7e

Vitalyk/multiturn v2 (NVIDIA#3167)

b9ee19e

ci: Disable the api check for now (NVIDIA#3157)

b168849

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

ci: Add DSv3 proxy (NVIDIA#3169)

a205538

Signed-off-by: oliver könig <okoenig@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

fdc04f6

[Community][Main] fix(moe): Fix theoretical memory calculation of lay…

9ad5906

…ernorm. (NVIDIA#2434) Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com>

fix: Set --refit-method default to gloo (NVIDIA#3172)

5415e1d

[fix] Bug fix for offloading in evaluate() (NVIDIA#3043)

a976754

Signed-off-by: Hongbin Liu <hongbinl@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

991c38f

cp: Fix: nccl-ub in ddp path (3181) into main (NVIDIA#3182)

5d0a7fd

Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com> Co-authored-by: Youngeun Kwon <youngeunk@nvidia.com>

Miscellaneous inference cleanup (NVIDIA#2955)

ffbc43f

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Revert "Miscellaneous inference cleanup (NVIDIA#2955)"

0fe3232

This reverts commit ffbc43f.

ci: Fix DSv3 (NVIDIA#3188)

69a5c63

Signed-off-by: oliver könig <okoenig@nvidia.com>

Fix missing argument in MoELayer.forward() (NVIDIA#3133)

2fadde8

Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com>

Fix H2D stream synchronization in optimizer offload (NVIDIA#3140)

ae67076

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Add MTP support for hybrid models (NVIDIA#2363)

300d1b6

Co-authored-by: Rabeeh Mahabadi <rkarimimahab@nb-hel-cs-001-vscode-02.cm.cluster> Co-authored-by: Sanjeev Satheesh <sasatheesh@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

docs: improve Megatron-LM and Megatron Core descriptions (NVIDIA#3115)

dceb1fb

Signed-off-by: Santosh Bhavani <santosh.bhavani@live.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Handle step key correctly in checkpoint save with `--optimizer-cpu-…

f4502eb

…offload` (NVIDIA#2874)

Merge commit 'f4502eb1c92f77e0ed190cac00293e8ac192543b' into rocm_dev

27f7be5

r0.16.0 IFU Cleanup

bdd74e2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Sudhu/megatron ifu core 0.16.0 sync#119

[WIP] Sudhu/megatron ifu core 0.16.0 sync#119
sudhu2k wants to merge 695 commits intorocm_devfrom
sudhu/Megatron-IFU-core_0.16.0_sync

sudhu2k commented Mar 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

sudhu2k commented Mar 3, 2026

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants