feat(forge): add coverage policies - #15959
Conversation
✅ Changelog foundThe deterministic check will validate the changed entry. |
Add an Istanbul-compatible JSON summary report for stable aggregate coverage data. Allow CLI and foundry.toml thresholds to fail coverage runs when line, statement, branch, or function coverage is too low.
bcc3c31 to
2ef6a4e
Compare
| cmd.args(["coverage", "--report=json-summary"]).assert_failure().get_output().clone(); | ||
| let stdout = output.stdout_lossy(); | ||
| let stderr = output.stderr_lossy(); | ||
| assert!(!stdout.contains("Wrote JSON summary report."), "{stdout}"); |
There was a problem hiding this comment.
Could we cover a mixed file-report invocation here too? LcovReporter and CoverageAttributionReporter still print their Wrote ... report. status via sh_println!, so combining json-summary with either leaks report status to stdout despite the PR's stated stderr contract. Switching those messages to sh_status! and updating the snapshot would keep all file reporters consistent.
| symbolic artifact replay, showmap replay, brutalization, or mutation testing. Use | ||
| `--report lcov` for interoperable coverage data or `--report attribution` for | ||
| Foundry's per-test JSON attribution report."#)] | ||
| `--report json-summary` for Istanbul-compatible summary JSON, `--report lcov` for |
There was a problem hiding this comment.
Could we also update TestArgs::ensure_coverage_mode_compatible? Its runtime error for rejected forge coverage --json still recommends only LCOV or attribution, omitting the new Istanbul-compatible --report json-summary replacement described here.
2 participants
Adds an explicit
json-summarycoverage report that writes deterministic, Istanbul-compatible aggregate and per-source counters tocoverage-summary.json, providing a stable CI interface without changing the meaning of global--json. This builds on #9525 and #15786 and delivers the coverage-policy use case raised in #3644.Adds independent integer thresholds for lines, statements, branches, and functions through
foundry.tomland--fail-under-*flags. Forge compares exact aggregate ratios, treats metrics with no coverable items as not applicable, writes requested reports before returning a nonzero status, and keeps report status text on stderr.foundry-rs/book#2026 documents the report schema, configuration, defaults, environment support, and CLI behavior. This PR was written with OpenAI Codex assistance, including investigation, implementation, tests, documentation, and the PR description.