Skip to content

Consolidate Python package management on uv - #8212

Merged
Eddy Ashton (eddyashton) merged 6 commits into
mainfrom
achamayou-investigate-pypi-cache
Aug 27, 2026
Merged

Consolidate Python package management on uv#8212
Eddy Ashton (eddyashton) merged 6 commits into
mainfrom
achamayou-investigate-pypi-cache

Conversation

@achamayou

@achamayou Amaury Chamayou (achamayou) commented Aug 26, 2026

Copy link
Copy Markdown
Member

Summary

Consolidates CCF's Python package management on uv instead of pip, across CI workflows, setup scripts, docs, sandbox, and tests.

  • Adds scripts/install_uv.sh, a pinned wrapper (uv 0.11.19) around the official uv installer script, which reuses a matching uv/uvx already on PATH when present.
  • Installs the wrapper into the install tree (CMake) and container/setup wiring, so it is available wherever CCF is installed.
  • Replaces pip installs/builds with uv/uv pip/uv build across CI workflows, scripts/setup-*.sh, the docs helper (livehtml.sh), the sandbox, tests, the release workflow, and Copilot instructions.
  • Creates venvs with python3 -m venv --without-pip where introduced by this change, since pip is no longer needed inside them.
  • Migrates workflow package-index configuration from PIP_INDEX_URL to UV_INDEX_URL, with local scripts bridging PIP_INDEX_URL to UV_INDEX_URL for compatibility where it was previously set externally.
  • Removes OS pip packages (python3-pip, python-pip) that are no longer needed once uv is used directly.

Out of scope

Caching of uv's Python package downloads across CI jobs is intentionally not part of this PR. That work is deferred to a follow-up PR stacked on top of this one: #8219.

Validation

  • Python SDK tests pass using the pinned uv bootstrap
  • Python wheel builds successfully with uv build --wheel
  • Official pinned installer, idempotence, and PATH reuse exercised
  • prettier-checks.sh -f, shellcheck-checks.sh, ascii-checks.sh, and copyright-checks.sh pass on the changed files

Copilot AI lite review requested due to automatic review settings August 26, 2026 13:41
@achamayou
Amaury Chamayou (achamayou) requested a review from a team as a code owner August 26, 2026 13:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the repo’s CI/dev/test Python dependency installation from pip to a pinned uv 0.11.19, and introduces a checksum-verified uv bootstrap script. It also adds GitHub Actions caching for uv’s content-addressed package cache (with separate scopes) and updates workflows to use UV_INDEX_URL rather than PIP_INDEX_URL.

Changes:

  • Replace pip install usage with uv pip install and create venv environments with --without-pip in CI and test/dev scripts.
  • Add scripts/install_uv.sh (pinned + SHA-256 verified) and install/package it for reuse in source and install trees.
  • Add uv cache configuration to the install-ci-dependencies composite action and update workflows to pass cache scope / disable cache where appropriate.

Custom instructions used:

  • .github/copilot-instructions.md

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/tests.sh Switch test venv setup to --without-pip, bootstrap uv, and install deps via uv pip.
tests/test_install.sh Use pip-free venv + installed-tree install_uv.sh, and install the Python package with uv pip.
tests/sandbox/sandbox.sh Use pip-free venv, bootstrap uv, and install sandbox deps via uv pip (source/install tree aware).
tests/recovery_benchmark.sh Add pipefail, use pip-free venv, bootstrap uv, and install locust via uv pip.
tests/ci/ccf_caci_ci Ensure install_uv.sh is available in the CI container build context.
scripts/setup-ubuntu-ci-checks.sh Replace pip-based uv install with pinned install_uv.sh.
scripts/setup-dev.sh Install pinned uv and use uv pip --system for Python tooling.
scripts/setup-dev-al4.sh Install pinned uv and use uv pip --system for Python tooling (plus clang-format pin).
scripts/setup-ci.sh Remove pip dependency and install pinned uv via install_uv.sh.
scripts/setup-ci-al4.sh Remove pip dependency and install pinned uv via install_uv.sh.
scripts/install_uv.sh New pinned, SHA-256-verified uv/uvx bootstrapper for x86_64 and aarch64.
livehtml.sh Switch local doc live-reload venv setup to uv + pip-free venv.
CMakeLists.txt Install scripts/install_uv.sh into the packaged bin/ directory.
.github/workflows/tla-shallow.yml Use UV_INDEX_URL instead of PIP_INDEX_URL.
.github/workflows/release.yml Use uv build --wheel for Python wheel builds and UV_INDEX_URL.
.github/workflows/README.md Document uv cache behavior and rationale.
.github/workflows/long-test.yml Use UV_INDEX_URL instead of PIP_INDEX_URL.
.github/workflows/doc.yml Use UV_INDEX_URL, pip-free venv + uv pip, and set python cache scope to docs.
.github/workflows/coverage.yml Use UV_INDEX_URL instead of PIP_INDEX_URL.
.github/workflows/codeql-analysis.yml Use UV_INDEX_URL and explicitly disable Python package caching for CodeQL.
.github/workflows/ci.yml Use UV_INDEX_URL, pip-free venv + uv pip, and set python cache scope to all.
.github/workflows/ci-verification.yml Use UV_INDEX_URL instead of PIP_INDEX_URL.
.github/workflows/ci-al4.yml Use UV_INDEX_URL instead of PIP_INDEX_URL.
.github/workflows/bencher.yml Use UV_INDEX_URL instead of PIP_INDEX_URL.
.github/workflows/bencher-ab.yml Use UV_INDEX_URL instead of PIP_INDEX_URL.
.github/copilot-instructions.md Update documented doc-build install command from pip to uv pip.
.github/actions/install-ci-dependencies/action.yml Add uv cache dir wiring and actions/cache restore for uv package cache with workload scopes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Amaury Chamayou (achamayou) force-pushed the achamayou-investigate-pypi-cache branch from f60162d to b46300c Compare August 26, 2026 13:59
Comment thread scripts/install_uv.sh Outdated
Revert all cache-related additions from the uv consolidation branch:
restore .github/actions/install-ci-dependencies/action.yml to main,
drop the python-cache-scope and cache-python-packages workflow inputs
from ci.yml, doc.yml and codeql-analysis.yml, and remove the uv
caching paragraph from .github/workflows/README.md. Also restore the
CCF_TEST_SYNC_AFTER_SETUP flush step in tests/tests.sh that had been
dropped by an earlier merge, unrelated to this change.

Caching of Python dependencies is intentionally deferred to a stacked
follow-up PR.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@achamayou Amaury Chamayou (achamayou) changed the title Cache Python dependencies with uv Consolidate Python package management on uv Aug 27, 2026
Merges current origin/main (e87931d) into this uv-only consolidation
branch so the PR's merge base advances past unrelated changes that
landed on main while this branch was being split out (notably the
CCF_TEST_SYNC_AFTER_SETUP tests/tests.sh change from #8215). This
ensures GitHub's three-dot diff for PR #8212 shows only the uv
package-management consolidation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@eddyashton
Eddy Ashton (eddyashton) merged commit 885794b into main Aug 27, 2026
21 checks passed
@eddyashton
Eddy Ashton (eddyashton) deleted the achamayou-investigate-pypi-cache branch August 27, 2026 13:07
Eddy Ashton (eddyashton) pushed a commit that referenced this pull request Aug 27, 2026
Restore uv's content-addressed package cache for CI, docs, and combined
workloads that was intentionally split out of #8212. Give uv a
writable cache directory outside /github/home/.cache, key the cache by
Azure Linux package manager, architecture, workload scope, and a hash
of the relevant dependency/check/setup files, rotate weekly, and
disable Python caching for the CodeQL job which installs no Python
packages.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Amaury Chamayou (achamayou) added a commit that referenced this pull request Aug 27, 2026
Restore uv's content-addressed package cache for CI, docs, and combined
workloads that was intentionally split out of #8212. Give uv a
writable cache directory outside /github/home/.cache, key the cache by
Azure Linux package manager, architecture, workload scope, and a hash
of the relevant dependency/check/setup files, rotate weekly, and
disable Python caching for the CodeQL job which installs no Python
packages.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Amaury Chamayou (achamayou) added a commit that referenced this pull request Aug 27, 2026
Restore uv's content-addressed package cache for CI, docs, and combined
workloads that was intentionally split out of #8212. Give uv a
writable cache directory outside /github/home/.cache, key the cache by
Azure Linux package manager, architecture, workload scope, and a hash
of the relevant dependency/check/setup files, rotate weekly, and
disable Python caching for the CodeQL job which installs no Python
packages.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants