Repository navigation
test: add UPDATE_GOLDEN flag to regenerate schema snapshots - #843
Conversation
The two golden-file tests (contacts + docs tool schemas) compare a
generated JSON schema against a committed snapshot and fail on drift,
but offered no way to refresh the snapshot after an intentional schema
change — contributors had to hand-edit the JSON.
Add a shared tests/golden_utils.assert_matches_golden() helper. When
UPDATE_GOLDEN=1 is set it rewrites the snapshot; otherwise it compares
and fails on drift exactly as before, now with a hint pointing at the
flag. Refactor both golden tests onto the helper (no behavior change in
default mode).
UPDATE_GOLDEN=1 uv run pytest
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/golden_utils.py`:
- Around line 43-46: The golden file I/O uses pathlib.Path.write_text/read_text
without an explicit encoding, which can vary by host locale; update the calls
around golden_path.write_text(_serialize(generated)) and golden =
json.loads(golden_path.read_text()) to pass encoding="utf-8" (i.e.,
golden_path.write_text(_serialize(generated), encoding="utf-8") and
json.loads(golden_path.read_text(encoding="utf-8"))) so reads/writes are
deterministic across environments.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 6f18a10c-38ef-462f-9601-21536b8d496a
📒 Files selected for processing (3)
tests/gcontacts/test_contacts_tools_v2.pytests/gdocs/test_strikethrough.pytests/golden_utils.py
…ructions The Testing & CI notes already mentioned diffing tool schemas against committed goldens, but not how to regenerate them. Name the UPDATE_GOLDEN=1 command so the workflow is discoverable proactively, complementing the failure-message hint added in this PR. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address CodeRabbit review on taylorwilsdon#843: Path.read_text/write_text default to the host locale encoding (e.g. cp1252 on Windows), which could make golden snapshots non-deterministic across environments. Pass encoding='utf-8' explicitly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks, this is a nice quality of life one! Merged main in and took a cleanup pass, one catch since you opened it - #1230 rewrote |
Description
Adds an opt-in
UPDATE_GOLDEN=1environment flag so the schema golden-file tests can regenerate their snapshots, instead of forcing contributors to hand-edit the JSON.Why this is needed
Two tests assert a generated tool schema against a committed snapshot and fail on any drift:
tests/gcontacts/test_contacts_tools_v2.py→golden/contacts_tool_schemas.jsontests/gdocs/test_strikethrough.py→golden/docs_tool_schemas.jsonThese guards are valuable — they catch accidental changes to a tool's public contract. But there was no supported way to update a snapshot after an intentional schema change. Any PR that legitimately changes a contacts/docs tool schema (adding a field, a parameter, etc.) makes these tests fail, and the only recourse is to manually reconstruct the exact
json.dumps(..., indent=2, sort_keys=True)output by hand — easy to get subtly wrong, and the failure message gave no guidance.This was hit in practice while adding a
birthdayfield to the contacts tool: the schema-golden test failed and had to be regenerated manually.What this does
tests/golden_utils.assert_matches_golden(generated, golden_path, label).UPDATE_GOLDEN=1→ the helper rewrites the golden file (canonicalindent=2, sort_keys=True+ trailing newline) and returns.Workflow for an intentional schema change becomes:
This is the standard snapshot pattern (pytest-snapshot's
--snapshot-update, Jest's-u, Rust insta'sINSTA_UPDATE). The diff still surfaces in code review, so the guardrail isn't weakened — it just stops being a manual transcription chore.Type of Change
Testing
Verified locally:
90 passedfor the two files;1199 passed, 2 skippedfull suite).UPDATE_GOLDEN=1regenerates the snapshots byte-identically when the schema is unchanged (no spurious diff).UPDATE_GOLDENhint; rerunning with the flag restores it.ruff checkandruff format --checkpass on all changed files (no unused imports left behind).Checklist
Additional Notes
Test-only change — no runtime/production code is touched. Independent of (and complementary to) the birthday feature in #833, which is what surfaced the gap.
Summary by CodeRabbit
Changes
valuealiases for email and phone inputs.Tests
Documentation