Improve tolerance for screenshot test comparisons #4173
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| final int toleratedChannelDelta = 2; | ||
| final double maxDifferentPixelsRatio = 0.001; // 0.1% of pixels may differ slightly | ||
| final double maxAverageChannelDelta = 1.5; // average difference per channel across entire image |
There was a problem hiding this comment.
With maxDifferentPixelsRatio set to 0.001 and maxAverageChannelDelta to 1.5, imagesWithinTolerance() will still pass when up to ~0.1% of pixels are completely wrong because the average delta across all channels stays far below 1.5. On a typical 1080p screenshot that means a 45×45 region (e.g., an icon disappearing) could differ by 255 per channel and the test would still report a match, undermining the screenshot regression check for any small UI element. Consider bounding the allowed per-pixel delta or tightening these thresholds so only minor noise is tolerated.
Useful? React with 👍 / 👎.
✅ Continuous Quality ReportTest & Coverage
Static Analysis
Generated automatically by the PR CI workflow. |
Android screenshot updatesCompared 7 screenshots: 6 matched, 1 updated.
Native Android coverage
|
iOS screenshot updatesCompared 7 screenshots: 5 matched, 2 updated.
|
This reverts commit 6d12f6c.
Summary
Testing
Codex Task