[NRT-873] Build the vendored bundle from uv.lock and guard Python 3.9 support - #1360
Merged
Conversation
RisingOrange
force-pushed
the
worktree-py39-bundle-guard
branch
2 times, most recently
from
August 5, 2026 11:56
9fa86f1 to
1b26344
Compare
RisingOrange
marked this pull request as ready for review
August 5, 2026 12:03
… support scripts/build.py re-resolved the bundle dependency group from pyproject.toml on every build. It did not read uv.lock, and it evaluated markers against the build machine's Python rather than the oldest interpreter the add-on has to run on, so any transitive that dropped Python 3.9 support reached users unnoticed - which is what happened when asgiref 3.12 was released. Anki bundles its own CPython. Official builds moved to 3.13 in 25.07, but 3.9 remained the minimum supported version through 25.09 and the builds before 25.07 do ship it, so with a declared minimum Anki of 2.1.56 the add-on still has to run there. Roughly a quarter of active add-on users are on an Anki predating 25.07. Install from uv.lock instead, in two layers, each with the oldest Python that can reach it: the bundle group with 3.9, and bundle_modern - protobuf-py, which has required 3.10 since its first release - with 3.10. uv then refuses a distribution declaring a newer floor, so the build fails rather than the add-on. The layers are dependency groups, so nothing routes on markers. Assert every exported requirement produced a distribution: a requirement whose marker is false for the building interpreter or platform installs nothing and reports nothing, so its absence is the only signal there is. This reads dist-info, so what it catches is a distribution that never arrived, not one whose files a prune removed while leaving its metadata. Assert the layers are disjoint, since they install into one directory in sequence. Clear ankihub/lib before installing and delete the previous .ankiaddon before re-zipping - uv pip install --target and zip both update in place. Vendor tzdata unconditionally. Django declares it win32-only, the build runs on Linux, and one artifact ships to every platform, so the marker dropped it for the only users who need it. Bump Django to 4.2.30, the last 4.2 security release. Drop playhouse's compiled extensions, which cannot load off the build host. Remove the sys.path entry for ankihub/lib/other, which has pointed at nothing since vendored dependencies stopped being committed. Resolution covers declared floors only, so scripts/check_addon_import.py imports the built add-on the way Anki does - aqt.addons first, then ankihub.entry_point - checks where each of the bundle's modules resolved from, and renders a template through the add-on's own Django setup. It runs in the CI 3.9 lane, the only place with an Anki-on-3.9 environment. The test suite cannot do this even though it runs on 3.9: pytest plugins and factory-boy import django, asgiref and urllib3 from the development environment before `import ankihub` prepends the bundle, so the bundled copies are shadowed there and never exercised. scripts/release.sh adds two checks over the extracted archive - required members present, and the 3.10-only layer importing under 3.10. Every workflow producing an artifact runs that script, so the AnkiWeb and S3 uploads and the PR test builds inherit them rather than needing their own.
RisingOrange
force-pushed
the
worktree-py39-bundle-guard
branch
from
August 5, 2026 12:11
1b26344 to
431ae7f
Compare
…t on 3.9 The prune named playhouse, but the hazard is not playhouse's: one artifact ships to every interpreter and platform Anki runs on, and the layered install compiles for 3.9 or 3.10 rather than the build machine's Python, so any extension that is not abi3 can only load by accident. Sweep the bundle instead - abi3 passes, playhouse's optional ones are dropped, anything else fails the build rather than being deleted quietly, since something may need it to import. check_addon_import.py reaches a module only if something imports it, so its coverage of a submodule is whatever the add-on's own import graph happens to touch: `asgiref.sync`, the module the incident was in, is covered because Django's template stack pulls it in, not because asgiref is vendored - `import asgiref` alone passes against a bundle carrying 3.12.1. Parse every .py in the bundle under 3.9 as well. That reaches every file, but it is the weaker signal and not what this rests on: it catches syntax a release started using, not the `from typing import ParamSpec` break, which parses on 3.9 and fails on import. Don't let an option line uv emits, such as --index-url, parse as a requirement named "-index-url" and fail the bundle-contents check with a confusing message. Unlink ankihub/lib when it is a symlink: worktrees set up for running Anki point it at the main checkout's copy, where rmtree raises an errno-less OSError and installing would write to the other checkout. Also check VERSION is in the archive, and record why the vendored urllib3 is 1.x - the dev group caps it, so nothing in the bundle group shows the reason.
abi3 answers only half the question. protobuf-py-ext's wheel is cp310-abi3-manylinux_2_17_x86_64: stable across CPython versions, and still unable to load on Windows, macOS or ARM. Letting the tag alone grant a pass would have waved through a future dependency whose extension is required, breaking every platform except the Linux builder, with no check firing. What makes an extension safe to vendor into a single cross-platform artifact is the package treating it as optional. Both of these do, so list them and fail on anything else: protobuf-py catches the failed import and falls back to pure Python, so its extension is kept for the platform it does load on, and nothing imports playhouse at all, so its 2 MB are dropped. Refuse a line in uv's export that is neither a comment, a --hash continuation, nor an option, instead of skipping it. Options add no distribution and stay skipped; an editable would, and silently leaves it outside every later check. Both found by Codex reviewing the branch.
The Python 3.9 import check needs an Anki running on 3.9, so it exists only in the CI lane that has one. But the AnkiWeb and S3 upload workflows are workflow_dispatch and call this script directly, so an artifact can be published without any 3.9 check having run. Parse the extracted bundle under 3.9 here. That narrows the gap rather than closing it: it catches syntax a release started using, not a 3.10-only import, which is the shape the original incident had. Invoke build.py as `uv run --locked`. A plain `uv run` syncs first and will re-lock a lockfile that has fallen behind pyproject.toml, after which build.py's own `uv export --locked` sees the lock it just wrote and is satisfied - so the artifact could be built from a resolution nobody reviewed. Delete temporary files with -exec rather than a pipe into xargs, which splits on whitespace: a vendored path containing a space became two unrelated arguments to `rm -rf`. -depth so a directory goes only after find has walked into it, and an escaped dot in the .pyc pattern so it cannot match a character it did not mean. Found by Codex reviewing the branch.
Cut what restated the code: a comment naming what OLDEST_MODERN_PYTHON already says, a docstring repeating its function's name, one announcing that load_addon loads the add-on. Compressed the rest to the fact a future editor could break - what uv export does with --locked, that uv's own output is not re-parsable, that a false marker installs nothing and reports nothing, that zip updates in place. Moved the note about the vendored urllib3 being held at 1.x to the dev group's own cap, which is what holds it. Attached to sentry-sdk in the bundle group it never reached the person who could loosen it. Dropped from the extension audit the platform it was confirmed on and the argument for failing rather than guessing - the commit and the error message carry those.
…all sites media_import ships inside the same artifact and reaches the same interpreters, but its requirements were installed with whatever Python ran the build - 3.13 in every workflow - so nothing refused a distribution that had dropped 3.9. They are not in uv.lock either; the submodule carries its own requirements.txt, and none of the checks reached them: the extension audit and the bundle-contents check are scoped to ankihub/lib, and so was the archive's parse pass. Install them with the oldest interpreter too, and parse them with the rest. Today the file holds one pinned pure-Python package, so nothing shipped was broken. Pass --locked at the two remaining call sites. setup-addon syncs and then runs build.py without it, so a stale lockfile is re-locked in passing and the CI lane named for checking Python 3.9 attests to a bundle release.sh would not build. Cap urllib3 in the bundle group rather than leaving it to the dev group's cap. This is not the Python floor the layers enforce - urllib3 2.x still supports 3.9 - but sentry-sdk 1.6.0 predates urllib3 2.0's API changes, and the bundled copy is never import-tested, since Anki preloads its own and the origin check requires the vendored one to lose. Re-locking adds the requirement and moves no versions. Remove the artifact when a check fails. It is zipped before it is checked, so a failure left a freshly-timestamped broken one at the repository root. All four found by Fable reviewing the branch.
abdnh
approved these changes
Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issues
asgiref<3.12Proposed changes
scripts/build.pyre-resolved thebundlegroup frompyproject.tomlon every build. It never readuv.lock, and it evaluated markers against the build machine's Python (3.13) rather than the oldest interpreter the add-on has to run on — so any transitive that dropped Python 3.9 reached users unnoticed, which is what happened whenasgiref3.12 was released.Anki bundles its own CPython. Official builds moved to 3.13 in 25.07, but the builds before that ship 3.9, and with a declared minimum Anki of 2.1.56 the add-on still has to run there. Roughly a quarter of active users are on an Anki predating 25.07.
The fix makes the break impossible rather than merely detected. The bundle now installs from the lockfile, in two layers, each with the oldest Python that can reach it: the
bundlegroup with 3.9,bundle_modern(protobuf-py, which has required 3.10 since its first release) with 3.10. uv refuses a distribution declaring a newer floor, so the build fails instead of the add-on. They are separate dependency groups, so routing never reads markers.Resolution only sees declared floors, so detection still covers the rest — an undeclared floor, a requirement a false marker dropped silently, a fault in assembling the archive — across the build, the packaged archive, and an import inside a real Anki on 3.9.
The build
uv.lock, not re-resolved.uv export --locked --only-group <group>intouv pip install --no-deps.--lockedalso fails the build when the lockfile is out of step withpyproject.tomlinstead of quietly preferring one. Side effect: mashumaro is now pinned to the revision the lock recorded rather than tracking the fork's mutablemaster.dist-info— catching a distribution that never arrived, not one whose files a prune removed.abi3fixes only the interpreter axis —protobuf-py-ext's wheel iscp310-abi3-manylinux_2_17_x86_64, unloadable on Windows, macOS or ARM. So what earns a place is the package falling back without it: protobuf-py catches the failed import (kept), nothing importsplayhouse(its 2 MB dropped). Anything else fails the build.ankihub/libis cleared and the previous.ankiaddondeleted —uv pip install --targetandzipboth update in place, so unvendored packages were surviving into local artifacts.media_importinstalls with 3.9 too. Its requirements come from the submodule's ownrequirements.txt, notuv.lock, and were installed with whatever Python ran the build — outside every check, all of which were scoped toankihub/lib. One pinned pure-Python package today, so nothing shipped was broken.What gets checked
scripts/check_addon_import.pyimports the built add-on the way Anki does —aqt.addonsfirst, thenankihub.entry_point— and asserts where each vendored top-level package resolved from. The list comes from the bundle directory, and each assertion's direction follows from whether Anki had already imported that name, so on 2.1.56certifi,typing_extensionsandurllib3must resolve outside the bundle and everything else inside it. It then renders a template through the add-on's ownrender_template_from_string(), covering packages nothing imports directly, such assqlparse. It also parses every.pyin the bundle under 3.9, reaching files nothing pulls in. Point it at a bundle carryingasgiref 3.12.1and it reproduces the reportedImportError. It runs in the CI 3.9 lane, the only place with an Anki-on-3.9 environment.scripts/release.shadds three cheap checks over the extracted.ankiaddon: required members present, parses under 3.9, and the 3.10-only layer imports under 3.10. These live there because every workflow producing an artifact runs that script, so one added later inherits them — previously only the GitHub release was covered, not the copy users install from AnkiWeb.build.pyis invoked asuv run --lockedhere and insetup-addon, since a plain sync or run re-locks a stale lockfile, after whichbuild.py's ownuv export --lockedis satisfied by what it just wrote.Why not the test suite. It runs the add-on on 3.9, but pytest plugins and factory-boy import django, asgiref and urllib3 from the dev environment before
import ankihubprepends the bundle, so the bundled copies are shadowed and never exercised — that lane vendored the brokenasgiref==3.12.1and went green. Reorderingconftest.pyfixes it, but the guarantee would then be an import order the import sorter is free to undo, failing silently.Smaller changes
urllib3inbundle. Not the Python floor the layers enforce — urllib3 2.x still supports 3.9 — but sentry-sdk 1.6.0 predates urllib3 2.0's API changes, and only thedevgroup's cap held it at 1.x.tzdataunconditionally. Django declares itwin32-only and the bundle is built on Linux, so the marker dropped it for exactly the users who need it. Latent until a template uses a date filter.uv.lockhunks are worth a look.sys.pathentry forankihub/lib/other— it has pointed at nothing since [INTP-141] Avoid comitting vendored dependencies #1176, while sitting ahead of the bundle.READMEandjustfilegain--group bundle_modern, without which a 3.10+ dev environment has noprotobuf-pyandmypyfails onankihub/proto/*.py.no_absolute_imports_from_ankihubskipsscripts/— the rule exists because Anki installs the add-on under its AnkiWeb id; build scripts are not shipped inside it.How to reproduce
GOOGLE_API_KEY=dummy bash ./scripts/release.sh # ends "... assembled, parses on Python 3.9, and its Python 3.10 layer loads" xvfb-run -a uv run --exact --group aqt_legacy --python 3.9 python scripts/check_addon_import.pyThe install-time refusal, which is what makes the original incident impossible:
uv pip install --python 3.9 --no-deps --target /tmp/x asgiref==3.12.1 # error: ... asgiref==3.12.1 depends on Python>=3.10 ... requirements are unsatisfiableAnd detection, for a bundle that acquired a bad version some other way:
uv pip install --python 3.13 --no-deps --target ankihub/lib asgiref==3.12.1 xvfb-run -a uv run --exact --group aqt_legacy --python 3.9 python scripts/check_addon_import.py # ImportError: cannot import name 'ParamSpec' from 'typing'Also verified: the artifact holds 15 distributions (one more than before — tzdata) at 7.2 MB compressed;
pytest ./tests/addon -m "not sequential and not performance"731 passed;mypyclean on 118 files.Verified by hand in Anki
The built artifact was installed and exercised in real Anki on three interpreters, since what changed is what the add-on ships:
django,asgiref.sync,peewee,structlog,sentry_sdkresolve from the bundle,urllib3from Anki's venv. Sync and deck install work, so peewee runs without the stripped C extensions. django-cotton renders end to end — component loader, attribute expressions, nested components — the extent of what 2.5.1 → 2.7.2 could have broken.urllib3resolves from Anki's site-packages, so the shadowing contract holds in a real host; theabi3extension loads.dlopenfails on the foreign x86_64 ELF, protobuf-py suppresses thatImportErrorand falls back to pure Python; messages still round-trip — the last known limit's fallback working with the wrong OS and architecture at once.Further comments
Known limits, so nobody assumes more coverage than exists:
entry_point.run(), which needs a running Anki, and it imports the package asankihubrather than under its AnkiWeb id.asgiref.sync— where the incident was — is covered because Django's template stack pulls it in, not becauseasgirefis vendored:import asgirefalone passes against a bundle carrying 3.12.1. The parse pass reaches every file but catches only syntax.release.sh's parse pass narrows the publish-path gap rather than closing it, for the same reason..sooutright rather than attempting the load.Known follow-ups, deliberately not here:
workflow_dispatchand have no CI gate, so a dispatch can publish without the 3.9 lane ever running.create_release.ymlhas the gate mechanism to copy.create_release.ymlchecks the last CI run's conclusion but not its commit, so a push whose run has not been created yet slips through. Affects all regressions, not just 3.9 ones.uv.lockcan yield a different wheel later via a different build backend.sentry-sdk1.6.0 (June 2022). It declares noRequires-Pythonand is what makes urllib3 a dependency at all.certifiis still vendored though shadowed, for the same reason urllib3 is. A size cleanup, not a 3.9 fix.create_release.ymlbuilds the artifact before committing and tagging the version bump. Left alone deliberately: reordering would leave a pushed tag for a release that never shipped whenever a check fails.