Make ingester and compactor decommission timeouts configurable - #6640
Conversation
9664b23 to
3615a19
Compare
3615a19 to
04abf8c
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3615a19d1e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // amount of time with which we can fully decommission an ingester. This will be | ||
| // most noticeable when using Quickwit locally. | ||
| Duration::from_secs(10) | ||
| Duration::from_secs(15) |
There was a problem hiding this comment.
Guard ingester timeouts below the drain delay
When ingest_api.decommission_timeout or QW_INGEST_DECOMMISSION_TIMEOUT is set below this 15s delay, shutdown will always time out before the ingester can even switch from Retiring to Decommissioning and close shards, even if the queues are already empty. That makes the new timeout knob ineffective for short Kubernetes termination grace periods; consider rejecting values at or below this delay or making the delay part of the configured shutdown budget.
Useful? React with 👍 / 👎.
9e514d8 to
8233cda
Compare
8233cda to
5de4895
Compare
Description
Per title.
How was this PR tested?
Added unit tests