fix(triggers): gate node_not_ready on sustained NotReady, not transition

[mayankpande88]

Problem

The node_not_ready matcher fired on every Ready: True→False transition. On clusters with spot/preemptible or autoscaled node pools, every node reclaim, scale-down, and graceful-shutdown drain flipped a node NotReady for ~30–60s right before it was deleted — each raising a HIGH issue for a node that no longer exists.

Confirmed in the dev DB: 3,347 node_not_ready events, 346 distinct nodes in 7 days, each firing once and never recovering under that name — the signature of lifecycle churn, not incidents. All on the all-spot k8s-dev runner pools.

Change

Make the matcher level-triggered on duration instead of edge-triggered on transition:

• Fires only once a node has continuously reported Ready=False for ≥15m (nodeNotReadyMinDuration). Reclaimed nodes are deleted before the window elapses → never fire. A genuinely stuck node still surfaces.
• Added RateLimit: 6h keyed on the per-episode fingerprint (name + lastTransitionTime), so the now level-triggered predicate doesn't re-fire on every kubelet heartbeat during a long outage. Recover-then-fail-again gets a new lastTransitionTime → new fingerprint → fires again.

Performance

Neutral-to-cheaper. The rate limiter sits before the expensive evidence-fetch path in the engine, so churn events stop at the predicate (cheaper than before, which proceeded to fingerprint + 3 K8s events-table API calls per transition). Rate-limiter entries are only created for nodes that pass the 15m gate — the churny ones never reach Allow, so no memory growth.

Notes

• Matches Ready=False only (unchanged). Unreachable nodes report Ready=Unknown and still won't fire — sustained-unreachable detection would be a follow-up.
• This stops new noise only; the existing OPEN node_not_ready backlog on k8s-dev is unaffected and can be cleared separately.

Test

Rewrote the matcher tests for the duration gate (fires past threshold, silent within threshold, silent when Ready). go build, go test ./pkg/triggers ./pkg/alerts, go vet all pass.

[gemini-code-assist]

Code Review

This pull request updates the nodeNotReadyMatcher to trigger only when a Node has been in a NotReady state for a sustained period (at least 15 minutes) instead of immediately on transition, which helps reduce noise from expected churn. Associated tests have been updated to verify this duration-based logic. The feedback suggests using time.RFC3339Nano instead of time.RFC3339 to parse Kubernetes timestamps, as the latter can fail when fractional seconds are present.

[github-actions]

📦 Image Tags Updated

I've automatically updated the image tags in `charts/nudgebee-agent/values.yaml` to the latest versions from GHCR for the `main` branch.

The image tags are now synchronized with the latest builds and ready for release.