feat: per-family drift notifications + stage-3 verification agent#457
Open
monsagri wants to merge 6 commits into
Open
feat: per-family drift notifications + stage-3 verification agent#457monsagri wants to merge 6 commits into
monsagri wants to merge 6 commits into
Conversation
Replace the single truncated drift-report blob with one summary message
plus one message per drifting family, posted individually to a generic
{title,body} Slack Workflow Builder trigger:
- summary: drift counts (zero categories dropped) + scaffoldable families
+ a run link;
- one per new endpoint family: paths + a link/CLI to dispatch the stage-2
scaffold workflow for that family;
- one per partial family with unclaimed operations: the lagging ops, flagged
as needing a manual PR (stage 2 only scaffolds new resources).
No tool change — driftreport already emits new_families / unclaimed_operations
/ status_counts in `-format json` and preserves exit-2-on-drift. Public-repo
safety preserved: family names / paths / operationIds go only to the private
Slack channel (passed to curl, never echoed); step summary stays yes/no.
Requires the SLACK_DRIFT_WEBHOOK_URL Workflow Builder trigger to declare two
Text variables `title` and `body` (was message/repo/run_url/report).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add verify-scaffold.yml: a workflow_dispatch(pr_number) job that verifies a
stage-2 scaffolded draft PR against a real LaunchDarkly account before a human
finishes it.
Flow: guard the PR (same-repo, scaffold/* branch, open, integer pr_number) →
check out the PR branch → fetch the LD token from SSM via AWS OIDC (the same
path the acceptance matrix uses; api_host forced to prod) → build the provider
to a pinned GOBIN with a terraform dev override → run claude-code-action to
review the code and exercise the new resource with real terraform plan/apply,
fixing functionality/tests until apply is clean. Applied example resources are
retained (no destroy), namespaced tf-verify-pr<N> and pre-cleaned per run.
Results post as a PR comment plus a "ready for review" message on the same
{title,body} Slack trigger as the drift report.
Gated by repository variable VERIFY_AGENT_ENABLED. Security: prereq cleanup is
a deterministic workflow step so the agent needs no curl with the live token;
git tooling is scoped to the needed subcommands; runs are serialized per PR.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire stage 1d (#458, merged to preview-v3) into the workflows on main. drift-report.yml: the summary tally now counts scaffoldable_resources and registered_candidates and lists candidate names alongside new families; a new per-candidate message announces each curated net-new resource in a partial family with a SCOPED stage-2 dispatch command (family + resource_name + operations). scaffold-resource.yml: new optional resource_name + operations dispatch inputs. In scoped mode the agent implements exactly one net-new resource covering the listed operations, branches/PRs under that name, and moves the ops out of the family's new_resource_candidates into a resources entry (the family stays partial) — explicitly NOT touching the family's existing resources. Whole-family mode is unchanged. verify-scaffold.yml needs no change: a scaffold/<resource_name> branch still matches its scaffold/* guard. actionlint + YAML clean; the notification loop was dry-run against synthetic scaffoldable_resources output. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two bugbot findings on this PR: - scaffold-resource.yml: SCOPED MODE (resource_name set) needs the operationId list, but nothing required it. Add a fail-fast guard step that errors when resource_name is set with empty operations, instead of letting the agent enter scoped mode with no ops and the whole-family slice. - verify-scaffold.yml: the pre-clean step only deletes the tf-verify-pr<N> PROJECT, so an account-scoped resource reusing a stable name would 409 on a re-run — contradicting the "overwritten each run" comment. Account-scoped resources now get a per-run name (tf-verify-pr<N>-run<R>) so re-applies don't collide, and the header comment is corrected to state that project-scoped resources are pre-cleaned/overwritten while account-scoped ones accumulate and need manual cleanup. actionlint + YAML clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 883a602. Configure here.
Bugbot: the Resolve PR step wrote head_branch/pr_url/pr_title to GITHUB_OUTPUT via plain `echo key=value`. A PR title (free text; a JSON string from `gh ... --json title` can contain a newline) could inject a second `head_branch=…` line. Actions takes the last duplicate key, so checkout could use a ref that never passed the scaffold/* guard. Strip CR/newlines from headRefName, url, and title right after extraction (so the guard also sees the sanitized ref). head/url are single-line by construction; title is the real vector. Verified: a title containing `\nhead_branch=attacker-branch` is flattened to one line, leaving exactly one head_branch output (the real branch). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

RUBBER STAMP - only edits to autogen pipeline that's not used in our
mainbranch/releaseExtends the API-coverage autogen pipeline (stages 1/2 shipped in #445/#446/#453) with two
main-side workflow additions. Both are workflow-YAML only — no provider/tool code changes — and follow the established pattern of living onmain(forworkflow_dispatchregistration) while operating on thepreview-v3line.1. Per-family drift notifications (
drift-report.yml)Replaces the single truncated drift blob with one summary message plus one message per drifting family, posted individually to a generic
{title, body}Slack Workflow Builder trigger:No tool change:
driftreport -format jsonalready emitsnew_families/unclaimed_operations/status_countsand preserves exit-2-on-drift. Public-repo safety preserved — detail goes only to the private Slack channel (passed tocurl, never echoed); the step summary stays yes/no.2. Stage-3 verification agent (
verify-scaffold.yml, new)workflow_dispatch(pr_number)→ verifies a stage-2 scaffolded draft PR against a real LaunchDarkly account before a human finishes it:guard the PR (same-repo,
scaffold/*, open) → check out the PR branch → fetch the LD token from SSM via AWS OIDC (same path as the acceptance matrix; api_host forced to prod) → build the provider to a pinnedGOBINwith a terraform dev override → runclaude-code-actionto review the code and exercise the new resource with realterraform plan/apply, fixing functionality/tests until apply is clean. Applied example resources are retained (nodestroy), namespacedtf-verify-pr<N>and pre-cleaned per run. Results post as a PR comment + a "ready for review" message on the same Slack trigger.Security: prereq cleanup is a deterministic workflow step (the agent needs no
curlwith the live token); git tooling is scoped to the needed subcommands; runs are serialized per PR.Validation
actionlintclean on both workflows; YAML parses.Operator prerequisites
SLACK_DRIFT_WEBHOOK_URLWorkflow Builder trigger reshaped to declare Text variablestitle+body(wasmessage/repo/run_url/report); both flows + the stage-3 ping use it.VERIFY_AGENT_ENABLED=true. The first run should be supervised (real applies under an admin token).Notes / residual
GITHUB_TOKENpushes don't re-trigger the PR's CI (re-run manually).tf-verify-pr<N>resources are retained for review and not auto-swept (one set per PR; pre-deleted on re-run).schedule:for the drift report and dropping theref: preview-v3checkout pins remain TODO for when v3 becomesmain.🤖 Generated with Claude Code
Note
High Risk
Stage 3 runs real terraform apply with a prod admin token and can push to PR branches; mis-guards or a bad dispatch could mutate live LD state or code outside intended scaffold PRs.
Overview
Drift reporting (stage 1) now emits JSON instead of markdown and replaces the single truncated Slack blob with multiple private-channel posts via a shared
{title, body}Workflow Builder webhook: a category summary, one message per new API family (paths + scaffold dispatch link/CLI), one per scaffoldable net-new resource in a partial family, and informational posts for unclaimed ops. Public logs still only show yes/no drift.Scaffold (stage 2) gains optional
resource_name/operationsinputs for scoped net-new resources, validation when scoped mode is incomplete, and agent prompts/branches/mapping instructions that follow whole-family vs scoped behavior.New
verify-scaffold.yml(stage 3) is aworkflow_dispatchflow that guards same-repo openscaffold/*PRs, loads a prod LD token via AWS OIDC/SSM, builds the provider with a pinnedGOBINterraform dev override, pre-deletestf-verify-pr<N>projects, runs a verification agent (real plan/apply, optional fixes pushed to the branch), and notifies Slack fromverify-result.json.Unrelated to autogen: generated
vercel-nativeintegration config adds an optionalurlfield.Reviewed by Cursor Bugbot for commit 5c81a79. Bugbot is set up for automated code reviews on this repo. Configure here.