Megatron --debug-rollout-only still enters actor train path and crashes on missing parallel_state

[mvillmow]

Miles rollout-only profiling bug in Megatron backend

Summary

qwen3-30B-A3B-05-rollout-nsys.sh successfully runs rollout generation and produces an
Nsight Systems report, but the job still exits with a failure afterward when using the
Megatron backend.

The generated report is valid, but the overall stage is marked failed because Miles still
falls into the actor training path in --debug-rollout-only mode.

Reproduction

From the RL360 repo root:

./scripts/profiling/qwen3-30B-A3B-05-rollout-nsys.sh

Observed in the March 18, 2026 run:

• rollout Nsight Systems capture completed successfully
• report was written to:

/workspace/logs/execution/qwen3-30B-A3B/20260318-193009/nsys/rollout_only.nsys-rep

• the job then failed with:

AttributeError: 'MegatronTrainRayActor' object has no attribute 'parallel_state'

Why this appears to be a Miles bug

In the Megatron backend:

• MegatronTrainRayActor.init() returns early when args.debug_rollout_only is set
• that means self.parallel_state is never initialized
• later, MegatronTrainRayActor.train() still executes preprocessing that accesses
  self.parallel_state

Relevant code paths in this repo mirror:

• miles/miles/backends/megatron_utils/actor.py
• miles/train.py

The failure happens after rollout data has already been generated, which is why:

• qwen3-30B-A3B-01-e2e-nsys.sh passes
• qwen3-30B-A3B-05-rollout-nsys.sh produces a usable .nsys-rep
• but the rollout-only stage still exits non-zero

Error snippet

ray.exceptions.RayTaskError(AttributeError): ray::MegatronTrainRayActor.train()
...
  File "/root/miles/miles/backends/megatron_utils/actor.py", line 291, in train
    rollout_data = get_rollout_data(self.args, rollout_data_ref, self.parallel_state)
AttributeError: 'MegatronTrainRayActor' object has no attribute 'parallel_state'

Current workaround

• treat the generated rollout_only.nsys-rep as valid profiling output
• ignore the stage failure for now
• prefer e2e or train_only stages when you need a clean green run

Suggested upstream fix

Guard the Megatron actor training path for debug_rollout_only before any access to
self.parallel_state, or ensure parallel_state is initialized even in rollout-only mode.

[mvillmow]

Miles version:
1f619e17587c8ffc56378c6ec67849a2df46d970
(submodule status reported as miles (remotes/origin/feat/dev_20251202-42-g1f619e17))

Hardware:

• 8x NVIDIA H200
• Driver: 570.133.20
• CUDA: 12.9

I’m not running a wrapper-specific workflow here. This reproduces by launching Miles directly with --debug-rollout-only under Nsight Systems in a 4-GPU Megatron + SGLang configuration.

Command shape (local paths replaced with placeholders):

nsys profile \
  --trace=cuda,nvtx,cublas,ucx,osrt \
  --cuda-trace-scope=process-tree \
  --wait=all \
  --trace-fork-before-exec=true \
  --cuda-memory-usage=true \
  --cuda-graph-trace=graph \
  --sample=process-tree \
  --cpuctxsw=process-tree \
  --output <path_to_output>/rollout_only \
  python3 /root/miles/train.py \
    --actor-num-nodes 1 \
    --actor-num-gpus-per-node 4 \
    --rollout-num-gpus 4 \
    --num-gpus-per-node 4 \
    --disable-bias-linear \
    --qk-layernorm \
    --group-query-attention \
    --num-attention-heads 32 \
    --num-query-groups 4 \
    --kv-channels 128 \
    --num-layers 48 \
    --hidden-size 2048 \
    --ffn-hidden-size 6144 \
    --normalization RMSNorm \
    --position-embedding-type rope \
    --norm-epsilon 1e-6 \
    --rotary-percent 1.0 \
    --rotary-base 10000000 \
    --swiglu \
    --untie-embeddings-and-output-weights \
    --vocab-size 151936 \
    --moe-ffn-hidden-size 768 \
    --moe-router-score-function softmax \
    --moe-token-dispatcher-type alltoall \
    --moe-router-topk 8 \
    --moe-layer-freq [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1] \
    --num-experts 128 \
    --moe-grouped-gemm \
    --moe-token-drop-policy probs \
    --moe-router-dtype fp32 \
    --moe-permute-fusion \
    --moe-aux-loss-coeff 0 \
    --hf-checkpoint <path_to_model> \
    --ref-load <path_to_model_torch_dist> \
    --prompt-data <path_to_file>.jsonl \
    --input-key prompt \
    --label-key label \
    --apply-chat-template \
    --rm-type deepscaler \
    --num-rollout 1 \
    --rollout-batch-size 1 \
    --n-samples-per-prompt 8 \
    --rollout-max-response-len 256 \
    --rollout-temperature 1 \
    --global-batch-size 8 \
    --optimizer adam \
    --lr 1e-6 \
    --lr-decay-style constant \
    --weight-decay 0.1 \
    --adam-beta1 0.9 \
    --adam-beta2 0.98 \
    --optimizer-cpu-offload \
    --overlap-cpu-optimizer-d2h-h2d \
    --use-precision-aware-optimizer \
    --advantage-estimator grpo \
    --use-kl-loss \
    --kl-loss-coef 0.00 \
    --kl-loss-type low_var_kl \
    --entropy-coef 0.00 \
    --eps-clip 0.2 \
    --eps-clip-high 0.28 \
    --tensor-model-parallel-size 4 \
    --sequence-parallel \
    --pipeline-model-parallel-size 1 \
    --context-parallel-size 1 \
    --expert-model-parallel-size 4 \
    --expert-tensor-parallel-size 1 \
    --recompute-granularity full \
    --recompute-method uniform \
    --recompute-num-layers 1 \
    --micro-batch-size 1 \
    --max-tokens-per-gpu 4096 \
    --rollout-num-gpus-per-engine 4 \
    --sglang-mem-fraction-static 0.7 \
    --sglang-cuda-graph-bs 1 2 4 8 \
    --attention-dropout 0.0 \
    --hidden-dropout 0.0 \
    --accumulate-allreduce-grads-in-fp32 \
    --attention-softmax-in-fp32 \
    --attention-backend flash \
    --debug-rollout-only

Observed behavior:

• rollout generation itself succeeds far enough to exercise SGLang decode
• after that, Miles still enters MegatronTrainRayActor.train()
• it then crashes because self.parallel_state was never initialized in debug_rollout_only mode

Traceback:

Traceback (most recent call last):
  File "/root/miles/train.py", line 101, in <module>
    train(args)
  File "/root/miles/train.py", line 81, in train
    ray.get(actor_model.async_train(rollout_id, rollout_data_ref))
...
  File "/root/miles/miles/backends/megatron_utils/actor.py", line 291, in train
    rollout_data = get_rollout_data(self.args, rollout_data_ref, self.parallel_state)
AttributeError: 'MegatronTrainRayActor' object has no attribute 'parallel_state'

This seems to happen because init() returns early in debug_rollout_only mode before self.parallel_state is created, but train() still assumes it exists.