Propagate suite-level CLI overrides to constituent task specs

[finbarrtimbers]

Summary

• When a -o CLI override targets a suite name, expand the suite and apply the override to every constituent task spec instead of dropping it.
• Non-suite task specs continue to receive overrides directly.

Example of the bug

olmo-eval run -m <model> -t aime_2022_to_2025 -o sampling_params.max_tokens=16384

Before this fix, the override is keyed by aime_2022_to_2025 but the runner only sees the expanded constituent specs (aime_2022:pass_at_32, aime_2023:pass_at_32, aime_2024:pass_at_32, aime_2025:pass_at_32), so the max_tokens=16384 override is silently dropped and each task runs with its default max_tokens. After the fix, the override is propagated to each child spec.

Test plan

• Invoke a suite with a CLI override (e.g. aime_2022_to_2025 -o sampling_params.max_tokens=16384) and confirm each constituent task receives the override.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2663b2e296

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Preserve priority suffix when propagating suite overrides

When a suite is invoked with a priority suffix, e.g. -t suite@high -o max_tokens=..., expand_tasks() later runs the children as child@high and _build_task_overrides() looks up overrides by that exact spec. This propagation stores the override under the bare child_spec, so the suite-level override is still silently dropped for prioritized suites; append the original priority suffix to each child override key to match the expanded run specs.

Useful? React with 👍 / 👎.