[codex] Fix RBench notebook root after generator move#229
Conversation
|
The path should be NVIDIA/cosmos/evaluation/cosmos3/Rbench. "cosmos3/Rbench" should be under "evaluation". |
|
@yaoxu-crypto great catch and fixed. |
|
It's still incorrect. Should be "/evaluation/cosmos3/Rbench", not "cosmos3/evaluation/Rbench" |
|
@yaoxu-crypto triple-check please? |
|
Now it's correct. But I just realized that Ming-Yu asked for subfolders under /evaluation/cosmos3/, which is "generator", "reasoner", and I believe also "action". See his comment on this doc: https://docs.google.com/document/d/1lEvHJugjCTHuSHgVwE00lsfF33SKVPw8WpX-TCwQ6WY/edit?tab=t.4909uyl57xqa |
|
Overall LGTM! Considering to be consistent with existing layout. Maybe the folder should be evaluation/cosmos3/generator/rbench? @yaoxu-crypto |
|
I think it makes more sense to be evaluation/cosmos3/action/rbench. But I'd like to defer it to you @mli0603 |
|
@lfengad @yaoxu-crypto changed to |
54dcccf to
ca98f2f
Compare
Summary
main, which already moves RBench toevaluation/cosmos3/generator/rbench.RBENCH_NOTEBOOK_ROOTso it resolves local assets from the new generator evaluation location.Validation
jq empty evaluation/cosmos3/generator/rbench/run_with_cosmos_framework.ipynbjq empty evaluation/cosmos3/generator/rbench/assets/prompts/*.jsonrg.mergeable: truefor PR [codex] Fix RBench notebook root after generator move #229.