Skip to content

Attempt to stabilize flaky benchmarks#3632

Open
nvtw wants to merge 3 commits into
newton-physics:mainfrom
nvtw:dev/tw4/benchmark_improvements
Open

Attempt to stabilize flaky benchmarks#3632
nvtw wants to merge 3 commits into
newton-physics:mainfrom
nvtw:dev/tw4/benchmark_improvements

Conversation

@nvtw

@nvtw nvtw commented Jul 24, 2026

Copy link
Copy Markdown
Member

Description

Stabilize SDF benchmarks by limiting gating coverage to resolutions with repeatable timings, and strengthen hydroelastic contact-surface repeatability coverage.

Checklist

  • New or existing tests cover these changes
  • The documentation is up to date with these changes
  • CHANGELOG.md has been updated (if user-facing change)

Test plan

uv run --extra dev -m newton.tests -k test_no_degenerate_triangles_deep_penetration

Summary by CodeRabbit

  • Bug Fixes

    • Improved stability and repeatability of hydroelastic contact-surface generation during deep penetration.
    • Ensured generated contact surfaces avoid degenerate triangles.
  • Benchmarks

    • Updated SDF performance benchmarks to use stable grid resolutions, excluding resolution 128 and higher.
  • Documentation

    • Updated the unreleased changelog entry with the related issue references.

nvtw added 3 commits July 23, 2026 12:59
Rebuild the deepest primitive contact case repeatedly and require a stable face count. Increase its contact capacity for the denser analytic box SDF and link the existing analytic primitive fix to newton-physics#3150.
@nvtw nvtw self-assigned this Jul 24, 2026
@nvtw
nvtw marked this pull request as draft July 24, 2026 06:58
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The hydroelastic deep-penetration test now checks repeatability across SDF rebuilds. The SDF benchmark excludes resolution 128 because of bimodal timing behavior, and the changelog references both related fixes.

Changes

Hydroelastic SDF stability

Layer / File(s) Summary
Deep-penetration repeatability checks
newton/tests/test_hydroelastic.py, CHANGELOG.md
The test repeats selected deep-penetration builds, checks contact-surface triangle degeneracy, and asserts stable face counts across rebuilds. The changelog adds the related fix reference.
SDF benchmark resolution coverage
asv/benchmarks/setup/bench_sdf.py
SDF build measurements now cover resolutions 32 and 64, excluding resolution 128 from batching and benchmark parameters.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the main goal of stabilizing flaky benchmark-related tests and benchmark measurements.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant