Copilot engine: spawn /usr/local/bin/copilot ENOENT on tool-cache hit (installer skips wrapper when GITHUB_PATH is set)

[strawgate]

Summary

On GitHub Actions runners that already have copilot-cli cached in the runner tool cache, Copilot-engine workflows fail at the Execute GitHub Copilot CLI step with:

[copilot-harness] pre-flight: command not found: /usr/local/bin/copilot (F_OK check failed - binary does not exist at this path)
[copilot-harness] attempt 1: failed to start process '/usr/local/bin/copilot': spawn /usr/local/bin/copilot ENOENT (code=ENOENT syscall=spawn /usr/local/bin/copilot)
[copilot-harness] attempt 1 failed: exitCode=1 failureClass=no_output ... attemptDurationMs=4

The harness always spawns the hardcoded absolute path /usr/local/bin/copilot, but on a tool-cache hit the installer only prepends the cached directory to PATH and never creates /usr/local/bin/copilot. On a fresh download it does create it, so the failure is flaky by runner: a runner with an in-window copilot-cli already cached fails; a runner that fresh-installs passes. This makes "just re-run it" unreliable.

Root cause

Verified at v0.86.1 (and the logic is identical in every release back through v0.83.4, and on main).

1. The compiler hardcodes the spawn path:

   • pkg/constants/constants.go:584 -> const CopilotBinaryPath = "/usr/local/bin/copilot"
   • emitted into every .lock.yml as ... copilot_harness.cjs /usr/local/bin/copilot ...

2. actions/setup/sh/install_copilot_cli.sh sets INSTALL_DIR="/usr/local/bin" (line 29). The fresh-download path installs there and matches CopilotBinaryPath:

   echo "Installing binary to ${INSTALL_DIR}..."                    # line 597
   maybe_sudo tar -xz -C "${INSTALL_DIR}" -f "${TEMP_DIR}/${TARBALL_NAME}"
   maybe_sudo chmod +x "${INSTALL_DIR}/copilot"                     # line 599

3. But the cache-hit path, activate_cached_copilot_bin(), returns early whenever GITHUB_PATH is set (i.e. always inside Actions), before the wrapper install:

   export PATH="${cached_copilot_dir}:$PATH"        # line 488
   if [ -n "${GITHUB_PATH:-}" ]; then
     echo "$cached_copilot_dir" >> "${GITHUB_PATH}"
     return 0                                        # line 494  <-- returns BEFORE the wrapper install
   fi
   # only reached when GITHUB_PATH is UNSET (outside Actions):
   maybe_sudo install -m 0755 "$wrapper_path" "${INSTALL_DIR}/copilot"   # line 505

So in Actions, a cache hit leaves copilot on PATH only (e.g. /opt/hostedtoolcache/copilot-cli/1.0.77/x64/bin/copilot), and /usr/local/bin/copilot never exists. The harness then spawns the hardcoded /usr/local/bin/copilot and gets ENOENT.

The harness has no PATH fallback: pre-flight does an F_OK check on the literal /usr/local/bin/copilot and fails immediately (attemptDurationMs=4, hasOutput=false).

Reproduction

• A Copilot-engine workflow run on a runner whose tool cache already has an in-window copilot-cli version (reproduces reliably on persistent-tool-cache self-hosted runners, e.g. Ubicloud-hosted Actions runners; GitHub-hosted ephemeral runners usually fresh-install and mask it).
• Install step logs:

  Selected best cached version: 1.0.77 at /opt/hostedtoolcache/copilot-cli/1.0.77/x64/bin/copilot
  Using cached GitHub Copilot CLI from /opt/hostedtoolcache/copilot-cli/1.0.77/x64/bin/copilot
    Prepended /opt/hostedtoolcache/copilot-cli/1.0.77/x64/bin to PATH
    Exporting /opt/hostedtoolcache/copilot-cli/1.0.77/x64/bin to GITHUB_PATH (...)
  

  (note: no "Installing binary to /usr/local/bin")
• Execute step: spawn /usr/local/bin/copilot ENOENT.

A fresh-download runner logs Skipping candidate (cache expired ...) -> Installing binary to /usr/local/bin... and succeeds, which is why the same workflow passes on some runners and fails on others.

Expected

The binary path the harness spawns should exist whether the installer fresh-downloaded or hit the tool cache.

Suggested fixes (any one)

• In activate_cached_copilot_bin(), install the ${INSTALL_DIR}/copilot wrapper before the GITHUB_PATH early return, so the hardcoded CopilotBinaryPath always resolves (symmetry with the fresh-download path).
• Or resolve the binary from PATH in the harness (command -v copilot) instead of the fixed CopilotBinaryPath.
• Or have the compiler pass the actually-resolved binary path to the harness rather than the constant.

Workaround (for anyone else hitting this)

A pre-agent-step (gh-aw runs these after the install step and before the agent) that symlinks the PATH-resolved binary into place:

pre-agent-steps:
  - name: Ensure Copilot CLI at the harness path
    run: |
      set -euo pipefail
      if [ ! -x /usr/local/bin/copilot ]; then
        target="$(command -v copilot || true)"
        [ -n "$target" ] && sudo ln -sf "$target" /usr/local/bin/copilot
      fi

This is a no-op on fresh-install runners and rescues the cache-hit case. Confirmed working: workflows that failed with the ENOENT above now log cache-hit path: linking /usr/local/bin/copilot -> /opt/hostedtoolcache/... and run to success.

Version: v0.86.1 (logic identical v0.83.4 .. main). Firewall / agent image version is irrelevant to this failure.