Honor server-provided repository head overrides by szgupta · Pull Request #13718 · warpdotdev/warp

szgupta · 2026-07-14T20:17:58Z

Description

Stack 1 of 2. The resolved repository state/reporting follow-up remains #15274 and will be restacked separately.

Accept sparse server-provided repository head overrides for task-backed agent runs and apply them while preparing environment repositories:

COMMIT_SHA resolves to an exact shallow detached checkout.
BRANCH performs a filtered, shallow, single-branch clone of the requested branch.
Repositories without a head override continue to clone their environment-configured default branch.

Each supplied repository head override must be unique and declared by the environment. Remotes are removed from overridden repositories after setup so the run cannot expand beyond the prepared repository state. This PR intentionally does not model or report resolved repository baselines.

The hidden CLI contract is --repository-head-override-json, backed by RepositoryHeadOverride and RepositoryHeadRef.

Linked Issue

N/A

Testing

./script/format
cargo test -p warp_cli (201 passed)
cargo test -p warp --lib repository_head_override (2 passed)
cargo test -p warp --lib commit_override (5 passed)
cargo test -p warp --lib branch_override (1 passed)
cargo clippy --workspace --exclude warp_completer --all-targets --tests -- -D warnings
cargo clippy -p warp_completer --all-targets --tests -- -D warnings
I have manually tested my changes locally with ./script/run (not applicable to this headless environment setup path)

Agent Mode

Warp Agent Mode - This PR was created via Warp's AI Agent Mode

CHANGELOG-NONE

Conversation: https://staging.warp.dev/conversation/68b43c0f-5ef2-4aa8-b911-115291bdb7a2

Co-Authored-By: Warp agent@warp.dev

oz-for-oss · 2026-07-14T20:18:11Z

@szgupta

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

oz-for-oss

Overview

This PR adds pinned repository baseline support for eval runs, including hidden CLI parsing, environment baseline validation, exact shallow detached checkouts, baseline reporting, remote removal after setup, and snapshot patching relative to pinned commits.

Concerns

No blocking correctness, security, or spec-alignment concerns found in the annotated diff.

Verdict

Found: 0 critical, 0 important, 0 suggestions

Approve

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

oz-for-oss · 2026-08-13T21:49:57Z

@szgupta

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

oz-for-oss

Overview

This PR adds pinned repository baselines to agent environment setup, wires baseline reporting through the harness-support client, and changes end-of-run snapshots to diff pinned repos against their server-provided baseline.

Concerns

Adding report_repository_baselines as a required HarnessSupportClient method leaves existing test client implementations outside this diff without the new method, so the test build will fail.
report_repository_baselines is enabled for every task-backed run, but environment prep treats report failures as fatal; that makes ordinary unpinned environments depend on a new endpoint before setup commands can run.

Verdict

Found: 0 critical, 2 important, 0 suggestions

Request changes

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

oz-for-oss · 2026-08-13T21:54:40Z

    /// artifacts already reported via `report_artifact`.
    async fn finish_task(&self, success: bool, summary: &str) -> Result<()>;
+    /// Report verified initial repository state before environment setup begins.
+    async fn report_repository_baselines(&self, repositories: &[RepositoryBaseline]) -> Result<()>;


⚠️ [IMPORTANT] Adding this required trait method leaves existing HarnessSupportClient test impls outside this diff, such as FailingUploadTargetsClient and RecordingClient in checkpoint_coordinator_tests.rs, without an implementation, so cargo test/clippy with tests will fail with E0046.

oz-for-oss · 2026-08-13T21:54:40Z

                    cloud_providers: Vec::new(),
                    environment: None,
+                    repository_baselines: args.repository_baselines.clone(),
+                    report_repository_baselines: args.task_id.is_some(),


⚠️ [IMPORTANT] This enables fatal repository-baseline reporting for every task-backed run, including unpinned environments; because prepare_environment_impl returns ReportRepositoryBaselines on any reporting failure, ordinary runs now abort before setup if the new endpoint is unavailable or transiently fails. Gate this to pinned runs or make unpinned reporting best-effort.

warp-agent-staging

Went through this part-by-part. Solid overall (thorough validation, real end-to-end test execution against git). A few findings below, none blocking.

warp-agent-staging · 2026-08-18T18:06:02Z

+  fi
+  printf '%s\n' "Cloning repository $repo_name..."
+  git clone --filter=tree:0 "$repo_url" "$target"
+}


[smell] clone_repo() here is duplicated verbatim from build_parallel_clone_command's clone_repo() (environment.rs:575-585). Consider sharing one implementation instead of copy-pasting.

warp-agent-staging · 2026-08-18T18:06:02Z

+        })
+        .await
+        .unwrap_or(ShellType::Bash)
+}


[complexity] Nice new helper, but the two pre-existing inline duplicates of this exact logic in clone_repos (~line 646-649) and clone_repo (~line 693-696) weren't refactored to call it. Since this file is already being touched, worth collapsing all three.

warp-agent-staging · 2026-08-18T18:06:02Z

+    pub repo_name: String,
+    pub commit_sha: String,
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub branch: Option<String>,


[correctness] branch is parsed/validated but never consumed — checkout_pinned_repo (environment.rs:423) always fetches by raw commit_sha only. Is this reserved for a follow-up, or should it feed the fetch/checkout logic?

warp-agent-staging · 2026-08-18T18:06:02Z

        match entry.kind {
            EntryKind::Repo => {
                repo_index += 1;
+                let baseline_sha = repository_baselines.get(Path::new(&entry.path));


[correctness] This matches by exact PathBuf equality against a map keyed by working_dir.join(&baseline.repo_name) (driver.rs), but entry.path comes from the external snapshot-declarations.sh script (warp-agent-docker, not this repo). Any path-normalization difference between the two silently drops the baseline (falls back to git diff HEAD) with no warning logged. Worth confirming the two are guaranteed to match, or adding a fallback/log on miss.

warp-agent-staging · 2026-08-18T18:06:02Z

+}
+
+#[test]
+fn pinned_prepare_command_rejects_shallow_repo_with_history_ref() {


[tests] The real-execution tests here (wrong-HEAD, non-shallow, exposed-history) all cover rejection paths. Worth adding one success-path test (correct shallow+detached+single-commit checkout) confirming valid pinned state doesn't false-fail — a trimmed version of this test with one commit + a .git/shallow write, asserting success instead.

cephalonaut

Posted some comments inline, overall looks great!

cephalonaut · 2026-08-18T17:34:42Z

+    format!("sh -c '{escaped_script}'")
+}
+
+async fn remove_pinned_remotes(


This is aggressive, but seems right for benchmarking. I guess we could make it optional if we wanted to make pinning a more generic capability some day.

Co-Authored-By: Warp <agent@warp.dev>

cla-bot Bot added the cla-signed label Jul 14, 2026

oz-for-oss Bot reviewed Jul 14, 2026

View reviewed changes

szgupta closed this Jul 18, 2026

szgupta reopened this Aug 13, 2026

oz-for-oss Bot reviewed Aug 13, 2026

View reviewed changes

szgupta force-pushed the suraj/oz-evals-pinned-git branch from 38b06ec to cdbb5f5 Compare August 18, 2026 17:15

szgupta mentioned this pull request Aug 18, 2026

Report repository HEADs at execution time #15274

Open

2 tasks

szgupta changed the title ~~Add pinned repository baselines for evals~~ Use server-provided repository baselines Aug 18, 2026

warp-agent-staging Bot reviewed Aug 18, 2026

View reviewed changes

cephalonaut approved these changes Aug 18, 2026

View reviewed changes

szgupta force-pushed the suraj/oz-evals-pinned-git branch from cdbb5f5 to 25b439b Compare August 18, 2026 19:21

szgupta changed the title ~~Use server-provided repository baselines~~ Honor server-provided repository checkout overrides Aug 18, 2026

szgupta force-pushed the suraj/oz-evals-pinned-git branch from 25b439b to d21e62f Compare August 19, 2026 04:31

szgupta changed the title ~~Honor server-provided repository checkout overrides~~ Honor server-provided repository head overrides Aug 19, 2026

Honor repository head overrides for agent runs

a1a56dc

Co-Authored-By: Warp <agent@warp.dev>

szgupta force-pushed the suraj/oz-evals-pinned-git branch from d21e62f to a1a56dc Compare August 19, 2026 04:56

Uh oh!

Conversation

szgupta commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Linked Issue

Testing

Agent Mode

Uh oh!

oz-for-oss Bot commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oz-for-oss Bot left a comment

Choose a reason for hiding this comment

Overview

Concerns

Verdict

Uh oh!

oz-for-oss Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oz-for-oss Bot left a comment

Choose a reason for hiding this comment

Overview

Concerns

Verdict

Uh oh!

oz-for-oss Bot Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

oz-for-oss Bot Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

warp-agent-staging Bot left a comment

Choose a reason for hiding this comment

Uh oh!

warp-agent-staging Bot Aug 18, 2026

Choose a reason for hiding this comment

Uh oh!

warp-agent-staging Bot Aug 18, 2026

Choose a reason for hiding this comment

Uh oh!

warp-agent-staging Bot Aug 18, 2026

Choose a reason for hiding this comment

Uh oh!

warp-agent-staging Bot Aug 18, 2026

Choose a reason for hiding this comment

Uh oh!

warp-agent-staging Bot Aug 18, 2026

Choose a reason for hiding this comment

Uh oh!

cephalonaut left a comment

Choose a reason for hiding this comment

Uh oh!

cephalonaut Aug 18, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

szgupta commented Jul 14, 2026 •

edited

Loading

oz-for-oss Bot commented Jul 14, 2026 •

edited

Loading

oz-for-oss Bot commented Aug 13, 2026 •

edited

Loading