CI Extend time limit for hypothesis tests#8346
Conversation
Add a comment explaining why there is no limit for CI jobs and how we detect that we are running in a CI.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe Hypothesis "unit" test profile's non-CI deadline in python/cuml/tests/conftest.py was increased from 2000ms to 5000ms, and comments were added documenting CI environment variable behavior affecting the deadline setting. ChangesHypothesis Deadline Configuration
Estimated code review effort: 1 (Trivial) | ~3 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
CI failure is probably due to NVIDIA/raft#3052 |
|
If its just one test I would consider tweaking or perf debugging the test. No hypothesis tests should be exceeding 5 seconds as its common to run hundreds of cases of that one test. |
csadorf
left a comment
There was a problem hiding this comment.
I agree with @RAMitchell that we should investigate when tests take significantly longer, but I know from experience that it is sometimes very hard to stay below this deadline for more complex strategies. I'm ok with increasing the deadline for now.
Running
pytest -ra -v test_logistic_regression.pywould reproducibly run into the time limit locally. So we extend the limit to 5s. As part of this we noticed that it is a bit opaque why and how the time limit is disabled when running in the CI, so this adds comments explaining this behaviour.closes #8312