Problem
Nothing builds or tests this repository on a pull request, and the test suite never runs in CI at all.
Current pull_request triggers:
| Workflow |
Trigger |
What it does |
claude-code-review.yml |
pull_request |
AI review, no build |
pr-title-check.yml |
pull_request_target |
conventional-commit title lint |
And the rest:
| Workflow |
Trigger |
Builds? |
Tests? |
release.yml |
push to main, dispatch |
yes |
no |
test-ghostty.yml |
workflow_dispatch only |
yes |
yes |
deploy-website.yml |
push to main (website/**), dispatch |
site only |
no |
So:
- A PR can be merged without anything ever compiling it.
release.yml builds but has no test step, so a green release does not imply passing tests.
- The XCTest suite (209 tests) only runs via
test-ghostty.yml, which is manual-dispatch only and exists to check compatibility against a new Ghostty tag, not to gate changes.
- The Python script tests under
scripts/test_*.py (18 tests covering changelog.py and appcast generation) run nowhere in CI.
Why it matters
The first time a change is compiled by CI is during the release build, on main, after merge. A compile error or failing test is therefore discovered at release time, and recovery means shipping another version rather than pushing a commit to a branch.
This is not hypothetical. 0.1.77 was tagged and published with no downloadable artifact because its release build failed, and 0.1.78 had to be cut to replace it. That particular failure was a bad submodule entry that any PR-level checkout would have caught immediately.
It also means correctness currently rests on whoever wrote the change having run ./scripts/dev.sh test locally, with nothing recording whether they did.
Proposal
Add a pull_request workflow that builds and tests. test-ghostty.yml already contains a working recipe (checkout with submodules, brew install xcodegen, ./.github/actions/install-zig, setup-bun, zig build the xcframework, xcodegen generate, then the FactoryFloorTests scheme).
Points worth deciding:
- Cost and time.
macos-15 minutes are expensive. The ghostty xcframework build dominates, but release.yml already caches it on ghostty-build-${{ hashFiles('ghostty/.git') }}, and caches created on main are readable from PR branches, so most PRs should hit it. Building -Dxcframework-target=aarch64-macos rather than universal would also cut it down for a test-only build.
- Scope. Could be limited with
paths to Sources/**, Tests/**, project.yml, Resources/** so documentation and website PRs stay fast, at the cost of the filter drifting out of date.
- Add a test step to
release.yml. Independently useful, so a release cannot be cut from a tree whose tests fail.
- Run the Python tests.
uvx pytest scripts/ on ubuntu-latest costs almost nothing and currently has no coverage at all.
- Formatting.
prek hooks (SwiftFormat, trailing whitespace, YAML/JSON checks) run locally on commit but are not enforced in CI, so a contributor without hooks installed can land unformatted code.
Happy to implement whichever combination you prefer.
Problem
Nothing builds or tests this repository on a pull request, and the test suite never runs in CI at all.
Current
pull_requesttriggers:claude-code-review.ymlpull_requestpr-title-check.ymlpull_request_targetAnd the rest:
release.ymlmain, dispatchtest-ghostty.ymlworkflow_dispatchonlydeploy-website.ymlmain(website/**), dispatchSo:
release.ymlbuilds but has no test step, so a green release does not imply passing tests.test-ghostty.yml, which is manual-dispatch only and exists to check compatibility against a new Ghostty tag, not to gate changes.scripts/test_*.py(18 tests coveringchangelog.pyand appcast generation) run nowhere in CI.Why it matters
The first time a change is compiled by CI is during the release build, on
main, after merge. A compile error or failing test is therefore discovered at release time, and recovery means shipping another version rather than pushing a commit to a branch.This is not hypothetical. 0.1.77 was tagged and published with no downloadable artifact because its release build failed, and 0.1.78 had to be cut to replace it. That particular failure was a bad submodule entry that any PR-level checkout would have caught immediately.
It also means correctness currently rests on whoever wrote the change having run
./scripts/dev.sh testlocally, with nothing recording whether they did.Proposal
Add a
pull_requestworkflow that builds and tests.test-ghostty.ymlalready contains a working recipe (checkout with submodules,brew install xcodegen,./.github/actions/install-zig,setup-bun,zig buildthe xcframework,xcodegen generate, then theFactoryFloorTestsscheme).Points worth deciding:
macos-15minutes are expensive. The ghostty xcframework build dominates, butrelease.ymlalready caches it onghostty-build-${{ hashFiles('ghostty/.git') }}, and caches created onmainare readable from PR branches, so most PRs should hit it. Building-Dxcframework-target=aarch64-macosrather thanuniversalwould also cut it down for a test-only build.pathstoSources/**,Tests/**,project.yml,Resources/**so documentation and website PRs stay fast, at the cost of the filter drifting out of date.release.yml. Independently useful, so a release cannot be cut from a tree whose tests fail.uvx pytest scripts/onubuntu-latestcosts almost nothing and currently has no coverage at all.prekhooks (SwiftFormat, trailing whitespace, YAML/JSON checks) run locally on commit but are not enforced in CI, so a contributor without hooks installed can land unformatted code.Happy to implement whichever combination you prefer.