unify overdue-prompt logic so the Sentry alert matches the dashboard#446
Merged
Conversation
The overdue Sentry alert judged a prompt by the most recent run across all its models (Math.max), while the dashboard and scheduler judge per-model and treat a model with no recorded run as overdue. Because failed runs write no prompt_runs row, a prompt broken on one provider but still running on another looked fresh to the alert, so it never fired even with hundreds of prompts shown overdue. Extract one per-model definition into @workspace/lib/overdue and use it in the dashboard, the scheduler's self-healing pass, and the alert. The alert keeps its 30-minute grace window (now a graceMs parameter); the dashboard uses no grace, so behavior there is unchanged. Also drop the stale "runs every 6 hours" comment — the schedule is every 5 minutes in the worker entrypoint.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A deployment had ~300 prompts shown overdue on the admin dashboard but got no Sentry alert, despite #442 shipping the overdue alert.
Root cause: the alert and the dashboard disagreed on what "overdue" means.
admin.ts,schedule-maintenance.ts) judge per-model: a prompt is overdue if any configured model is past cadence or has never run.reportOverduePrompts) judged per-prompt offMath.max(...runTimes)— the single most-recent run across all models.Failed runs write no
prompt_runsrow (savePromptRunonly runs on success), so a broken model is simply absent from the last-runs map. The dashboard flags that prompt; the alert'sMath.maxpicks a healthy model's fresh timestamp and sees nothing wrong. When every overdue prompt is the "one provider down, others fine" kind, the alert computes0and returns before ever calling Sentry.Change
Extract one per-model definition into
@workspace/lib/overdueand route all three consumers through it:admin.ts)getModelOverdueStatus(grace0) — behavior identicalisPromptOverdue(grace0) — behavior identicalreportOverduePrompts)Math.maxacross all modelsisPromptOverdue(grace30m) — now matches the dashboardThe alert keeps its 30-minute grace window (now a
graceMsparameter) so jitter and freshly-created prompts don't page; the dashboard passes no grace, so its numbers are unchanged. The only remaining gap is the boundary — a prompt overdue by <30 min shows on the dashboard but doesn't page.Also removed the stale
Runs every 6 hoursdoc comment; the real schedule is*/5 * * * *in the worker entrypoint.Notes
SENTRY_DSNis set in the worker's environment (optional; without it the worker skipsSentry.init()and every capture is a no-op).Test plan
apps/web+apps/workertsc --noEmitclean@workspace/libsuite green, incl. newoverdue.test.tscovering the fresh-on-one-model / never-run-on-another case and the dashboard-equivalence (grace0)