Skip to content

fix(verify): tests check runs the target project's own interpreter - #105

Open
kylewelsby wants to merge 1 commit into
Cranot:mainfrom
kylewelsby:fix/verify-tests-project-interpreter
Open

fix(verify): tests check runs the target project's own interpreter#105
kylewelsby wants to merge 1 commit into
Cranot:mainfrom
kylewelsby:fix/verify-tests-project-interpreter

Conversation

@kylewelsby

Copy link
Copy Markdown

Summary

  • _run_impacted_pytest (the tests check inside roam verify) always shelled out with sys.executable. Under the documented uv tool install / pipx install path, that's roam's own isolated interpreter — it has neither pytest nor the target project's dependencies installed, so collection fails before any test runs, and the gate reports it as a hard-blocking failure: pytest exited 1 without a parsed passing result.
  • This hits any project whose dependencies don't happen to already exist inside roam's own venv, which in practice is close to every real project using the recommended global install.
  • The same bug class was already fixed in this release for the pre-push hook ("ran whatever python PATH resolved to, not the project's interpreter") — this closes the equivalent gap in verify's own tests check, which wasn't touched by that fix.

Fix

New _resolve_test_interpreter() walks up from the first impacted test file (not just root) looking for a .venv/venv, and falls back to sys.executable unchanged when none is found — so today's behaviour is preserved whenever roam is already running inside the project's own venv. Anchoring on the test file rather than root also resolves correctly for a monorepo subproject nested below the indexed root (e.g. a Python service in an otherwise non-Python repo), where the project-root directory itself has no venv at all.

Test plan

  • 3 new regression tests in tests/test_verify_autofire_hardening.py: prefers the project's own root venv, prefers a venv nested under the target file over a root-level one, falls back to sys.executable when no project venv exists.
  • uv run pytest tests/test_verify_autofire_hardening.py -v — 18/18 pass.
  • uv run pytest tests/test_verify*.py -p no:randomly — 256/256 pass, run twice for determinism. (One flake surfaced under random ordering; reproduced identically on unpatched main, so it's pre-existing pytest-randomly + shared index-state flakiness, unrelated to this change.)
  • uv run ruff check / uv run ruff format --diff on both changed files — clean.
  • Manually reproduced the original bug end to end against a real target repo (Rails + a nested Python subproject with its own .venv) before writing the fix, and confirmed the patched build resolves and runs the correct interpreter there.

Searched issues, PRs and discussions on this repo first — found no prior report of this.

_run_impacted_pytest always shelled out with sys.executable, which is
roam's own interpreter when installed isolated from any one project
(uv tool install, pipx — the documented install path). That
interpreter has neither pytest nor the target project's dependencies,
so collection fails before any test runs: "pytest exited 1 without a
parsed passing result", hard_block: true, on every project whose
dependencies roam's own venv doesn't happen to already have.

Resolve the interpreter from a .venv/venv found by walking up from the
first impacted test file instead, falling back to sys.executable
unchanged when no project venv is found. Walking from the test file
rather than root also picks the right interpreter for a monorepo
subproject nested below the indexed root.

Same bug class already fixed for the pre-push hook (this release,
"ran whatever python PATH resolved to, not the project's
interpreter") — this closes the same gap in `roam verify`'s own
tests check.
@kylewelsby
kylewelsby requested a review from Cranot as a code owner August 21, 2026 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant