Skip to content

[codex] Fix RBench notebook root after generator move#229

Open
mli0603 wants to merge 1 commit into
mainfrom
codex/move-rbench
Open

[codex] Fix RBench notebook root after generator move#229
mli0603 wants to merge 1 commit into
mainfrom
codex/move-rbench

Conversation

@mli0603

@mli0603 mli0603 commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Rebase onto latest main, which already moves RBench to evaluation/cosmos3/generator/rbench.
  • Update the RBench notebook RBENCH_NOTEBOOK_ROOT so it resolves local assets from the new generator evaluation location.

Validation

  • jq empty evaluation/cosmos3/generator/rbench/run_with_cosmos_framework.ipynb
  • jq empty evaluation/cosmos3/generator/rbench/assets/prompts/*.json
  • Confirmed no stale old-location notebook/path references with rg.
  • GitHub reports mergeable: true for PR [codex] Fix RBench notebook root after generator move #229.

@mli0603 mli0603 marked this pull request as ready for review June 22, 2026 16:33
@yaoxu-crypto

Copy link
Copy Markdown
Collaborator

The path should be NVIDIA/cosmos/evaluation/cosmos3/Rbench. "cosmos3/Rbench" should be under "evaluation".

@mli0603

mli0603 commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator Author

@yaoxu-crypto great catch and fixed.

@mli0603 mli0603 changed the title [codex] Move Rbench under cosmos3 [codex] Move RBench under cosmos3 evaluation Jun 22, 2026
@yaoxu-crypto

Copy link
Copy Markdown
Collaborator

It's still incorrect. Should be "/evaluation/cosmos3/Rbench", not "cosmos3/evaluation/Rbench"

@mli0603 mli0603 changed the title [codex] Move RBench under cosmos3 evaluation [codex] Move RBench under evaluation cosmos3 Jun 22, 2026
@mli0603

mli0603 commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator Author

@yaoxu-crypto triple-check please?

@yaoxu-crypto

Copy link
Copy Markdown
Collaborator

Now it's correct. But I just realized that Ming-Yu asked for subfolders under /evaluation/cosmos3/, which is "generator", "reasoner", and I believe also "action". See his comment on this doc: https://docs.google.com/document/d/1lEvHJugjCTHuSHgVwE00lsfF33SKVPw8WpX-TCwQ6WY/edit?tab=t.4909uyl57xqa
Maybe let's wait for him to confirm that we also need an "action" folder?

@lfengad

lfengad commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Overall LGTM! Considering to be consistent with existing layout. Maybe the folder should be evaluation/cosmos3/generator/rbench? @yaoxu-crypto

@yaoxu-crypto

Copy link
Copy Markdown
Collaborator

I think it makes more sense to be evaluation/cosmos3/action/rbench. But I'd like to defer it to you @mli0603

@mli0603 mli0603 changed the title [codex] Move RBench under evaluation cosmos3 [codex] Move RBench under evaluation cosmos3 generator Jun 25, 2026
@mli0603

mli0603 commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator Author

@lfengad @yaoxu-crypto changed to evaluation/cosmos3/generator/rbench, as this is a generator capability for video.

@mli0603 mli0603 force-pushed the codex/move-rbench branch from 54dcccf to ca98f2f Compare June 25, 2026 05:16
@mli0603 mli0603 changed the title [codex] Move RBench under evaluation cosmos3 generator [codex] Fix RBench notebook root after generator move Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants