Skip to content

[Tinker API] Restore variable-length result order - #2043

Open
yapdianang wants to merge 1 commit into
NovaSky-AI:mainfrom
yapdianang:dydx/native-gspo-tinker
Open

[Tinker API] Restore variable-length result order#2043
yapdianang wants to merge 1 commit into
NovaSky-AI:mainfrom
yapdianang:dydx/native-gspo-tinker

Conversation

@yapdianang

@yapdianang yapdianang commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Return each variable-length training result in the same order as its input example.
  • Remove results produced only for padding.

Before

Token-based batching returned results in packed microbatch order. A client could pair one example's mask with another example's log probabilities and fail before the optimizer step.

After

SkyRL maps each result back to the original example index before it returns loss_fn_outputs. The change applies to Trajectory custom GSPO and any other client that reads per-example results.

Testing

uv run --isolated --extra dev --extra tinker --extra skyrl-train \
  pytest tests/backends/skyrl_train/test_token_based_batching_utils.py -q

18 passed.

@yapdianang
yapdianang force-pushed the dydx/native-gspo-tinker branch from d900e34 to e2dcd27 Compare August 14, 2026 23:37
@yapdianang yapdianang changed the title [Tinker] Execute GSPO in one native forward-backward pass [Tinker] Execute GSPO in one ordered native forward-backward pass Aug 17, 2026
@yapdianang
yapdianang force-pushed the dydx/native-gspo-tinker branch from 8e9a895 to 20bd159 Compare August 17, 2026 19:04
@yapdianang yapdianang changed the title [Tinker] Execute GSPO in one ordered native forward-backward pass [Tinker API] Prune and order native GSPO updates Aug 17, 2026
@yapdianang
yapdianang force-pushed the dydx/native-gspo-tinker branch from 20bd159 to 614d03c Compare August 18, 2026 20:56
@yapdianang
yapdianang marked this pull request as ready for review August 18, 2026 23:02

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for the GSPO loss function, including batch normalization, row selection, and pruning metrics. It also implements sequence-based scheduling using sequence IDs to ensure requests are processed in order, alongside corresponding API validation and unit tests. The review feedback highlights three key issues: a potential ValueError when zipping outputs if per_sample_outputs is empty, a potential permanent hang during sequence bootstrapping if the database is reset and a non-zero sequence ID is used, and a potential AttributeError if metrics is None in pipeline parallel setups.

Comment thread skyrl/backends/skyrl_train_backend.py Outdated
Comment thread skyrl/tinker/engine.py Outdated
Comment thread skyrl/backends/skyrl_train/workers/megatron/megatron_worker.py
@yapdianang yapdianang changed the title [Tinker API] Prune and order native GSPO updates [Tinker API] Preserve input order for native GSPO updates Aug 18, 2026

@erictang000 erictang000 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some initial comments, need to look closer at the seq_id/engine logic changes as well!

Comment thread skyrl/backends/skyrl_train_backend.py Outdated
Comment thread skyrl/backends/skyrl_train_backend.py Outdated
Comment thread skyrl/backends/skyrl_train_backend.py Outdated
@yapdianang
yapdianang force-pushed the dydx/native-gspo-tinker branch from 150c2d8 to 0e3f075 Compare August 19, 2026 03:21

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

Reviewed by Cursor Bugbot for commit 0e3f075. Configure here.

Comment thread skyrl/tinker/engine.py Outdated
Signed-off-by: Dian Ang <23232359+yapdianang@users.noreply.github.com>
@yapdianang
yapdianang force-pushed the dydx/native-gspo-tinker branch from fdc3307 to ee28351 Compare August 26, 2026 21:02
@yapdianang yapdianang changed the title [Tinker API] Preserve input order for native GSPO updates [Tinker API] Restore variable-length result order Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants