Skip to content

feat: per-family drift notifications + stage-3 verification agent #457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
monsagri merged 6 commits into from
Jun 16, 2026
Merged

feat: per-family drift notifications + stage-3 verification agent #457

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1cf8fac
feat: per-family drift Slack notifications (autogen stage 1c)
monsagri Jun 15, 2026
996c4c9
feat: scaffolded-resource verification agent (autogen stage 3)
monsagri Jun 15, 2026
9fb6b45
feat: surface scaffoldable resources + scoped stage-2 scaffolding
monsagri Jun 16, 2026
b101519
make generate
monsagri Jun 16, 2026
883a602
fix: guard scoped scaffold inputs + account-scoped verify re-runs
monsagri Jun 16, 2026
5c81a79
fix: sanitize gh PR fields before writing GITHUB_OUTPUT
monsagri Jun 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 109 additions & 40 deletions .github/workflows/drift-report.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,21 @@
#
# Diffs the endpoint families of LaunchDarkly's public OpenAPI spec against the
# resources registered in the provider, using the curated mapping at
# scripts/driftreport/mapping.yaml. On drift, posts to a private Slack channel
# scripts/driftreport/mapping.yaml. On drift, posts to a PRIVATE Slack channel
# via the SLACK_DRIFT_WEBHOOK_URL secret. This repo is PUBLIC, so internal
# drift tracking must not use GitHub issues.
# drift tracking must not use GitHub issues, artifacts, or step-summary detail.
#
# NOTIFICATION MODEL (one Slack message becomes several):
# 1. a summary message (drift counts per category), then
# 2. one message per NEW endpoint family, each carrying the family's paths and
# a one-click link + CLI to dispatch the stage-2 scaffold workflow for it,
# 3. one message per partial family with UNCLAIMED operations (informational —
# stage 2 only scaffolds NEW resources, so these need a manual PR).
# Messages are POSTed individually to a SINGLE generic Slack Workflow Builder
# webhook trigger. That trigger must declare exactly two Text variables —
# `title` and `body` — and a "Send a message" step into the private channel.
# (`body` is Slack mrkdwn: <url|label> links render.) The stage-3 verify
# workflow reuses the same trigger for its "ready for review" message.
name: API drift report

on:
Expand Down Expand Up @@ -42,7 +54,10 @@ jobs:
id: drift
run: |
set +e
/tmp/driftreport -out drift-report.md
# JSON (not md): the notify step builds Slack messages from the
# structured report. The file stays on the ephemeral runner and is
# never uploaded or printed — it carries the internal roadmap.
/tmp/driftreport -format json -out drift-report.json
code=$?
set -e
if [ "$code" -ne 0 ] && [ "$code" -ne 2 ]; then
Expand All @@ -51,46 +66,100 @@ jobs:
fi
drift=$([ "$code" -eq 2 ] && echo true || echo false)
echo "drift=$drift" >> "$GITHUB_OUTPUT"
# PUBLIC-REPO SAFETY: this run's logs/summary are world-readable, so the
# report body must NOT land here. Summary states only whether drift was
# found; the detail goes to the private Slack channel. drift-report.md
# stays on the ephemeral runner (never uploaded as an artifact).
# PUBLIC-REPO SAFETY: this run's logs/summary are world-readable, so
# only the yes/no fact lands here. Detail goes to the private channel.
if [ "$drift" = "true" ]; then
echo "Drift detected — full report sent to the private Slack channel." >> "$GITHUB_STEP_SUMMARY"
echo "Drift detected — detail sent to the private Slack channel." >> "$GITHUB_STEP_SUMMARY"
else
echo "No drift detected." >> "$GITHUB_STEP_SUMMARY"
fi
# Build the Slack payload with jq so the report markdown is correctly
# JSON-escaped (quotes/newlines/backticks can't break the payload).
# Truncated to stay under Slack's ~4000-char webhook-variable limit;
# full detail is one re-dispatch away.
- name: Build Slack payload
id: slack_payload
if: steps.drift.outputs.drift == 'true'
run: |
limit=3500
report="$(head -c "$limit" drift-report.md)"
if [ "$(wc -c < drift-report.md)" -gt "$limit" ]; then
report="${report}"$'\n\n…(truncated — re-dispatch the workflow for the full report)'
fi
payload="$(jq -nc \
--arg message "API coverage drift detected — the provider lags behind the public API. Review and decide whether to scaffold." \
--arg repo "$GITHUB_REPOSITORY" \
--arg run_url "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" \
--arg report "$report" \
'{message: $message, repo: $repo, run_url: $run_url, report: $report}')"
{
printf '%s\n' 'payload<<SLACK_PAYLOAD_EOF'
printf '%s\n' "$payload"
printf '%s\n' 'SLACK_PAYLOAD_EOF'
} >> "$GITHUB_OUTPUT"
# Notify a private Slack channel on drift. The Workflow Builder trigger
# behind SLACK_DRIFT_WEBHOOK_URL should declare these Text variables:
# message, repo, run_url, report.
# Post one summary + one-per-family message to the private Slack channel.
# Each POST carries {title, body}; the Workflow Builder trigger renders
# them. PUBLIC-REPO SAFETY: family names / paths / operationIds are the
# internal roadmap — they are passed only to curl, NEVER echoed to the log.
- name: Notify Slack on drift
if: steps.drift.outputs.drift == 'true'
uses: slackapi/slack-github-action@485a9d42d3a73031f12ec201c457e2162c45d02d # v2.0.0
with:
webhook: ${{ secrets.SLACK_DRIFT_WEBHOOK_URL }}
webhook-type: webhook-trigger
payload: ${{ steps.slack_payload.outputs.payload }}
shell: bash
env:
WEBHOOK: ${{ secrets.SLACK_DRIFT_WEBHOOK_URL }}
REPO: ${{ github.repository }}
SERVER: ${{ github.server_url }}
RUN_ID: ${{ github.run_id }}
run: |
JSON=drift-report.json
RUN_URL="$SERVER/$REPO/actions/runs/$RUN_ID"
ACTIONS_URL="$SERVER/$REPO/actions/workflows/scaffold-resource.yml"
LIMIT=3500
sent=0

# post_slack <title> <body>: POST one message to the Workflow Builder
# trigger. Truncates body to Slack's webhook-variable limit. Never
# echoes title/body (roadmap) to the world-readable log.
post_slack() {
local title="$1" body="$2" payload
if [ "${#body}" -gt "$LIMIT" ]; then
body="${body:0:$LIMIT}"$'\n\n…(truncated — re-dispatch for full detail)'
fi
payload="$(jq -nc --arg title "$title" --arg body "$body" '{title: $title, body: $body}')"
curl -sS -f --retry 3 --retry-connrefused --retry-delay 2 \
-X POST -H 'Content-Type: application/json' -d "$payload" "$WEBHOOK" >/dev/null
sent=$((sent + 1))
}

# 1) Summary — one tally line (zero categories dropped) + scaffoldable
# families + run link. The title carries the headline, so the body
# doesn't restate it.
summary_body="$(jq -r --arg run "$RUN_URL" '
def lbl($n; $one; $many): if $n > 0 then "\($n) \(if $n == 1 then $one else $many end)" else empty end;
([ lbl(.new_families | length; "new family"; "new families"),
lbl(.scaffoldable_resources | length; "scaffoldable resource"; "scaffoldable resources"),
lbl(.unclaimed_operations | length; "unclaimed op"; "unclaimed ops"),
lbl(.stale_families | length; "stale family"; "stale families"),
lbl(.stale_operations | length; "stale op claim"; "stale op claims"),
lbl(.unmapped_resources | length; "unmapped resource"; "unmapped resources"),
lbl(.registered_candidates | length; "miscurated candidate"; "miscurated candidates")
] | join(" · ")) +
(([.new_families[].tag] + [.scaffoldable_resources[].name]) as $s
| if ($s | length) > 0 then "\nScaffoldable: " + ($s | join(", ")) else "" end) +
"\n<\($run)|drift run>"
' "$JSON")"
post_slack "🔎 API coverage drift" "$summary_body"

# 2) One message per NEW family — scaffoldable via stage 2. Paths inline,
# scaffold link + CLI on one line.
while IFS= read -r fam; do
[ -n "$fam" ] || continue
tag="$(printf '%s' "$fam" | jq -r '.tag')"
paths="$(printf '%s' "$fam" | jq -r '[.paths[] | "`" + . + "`"] | join(" · ")')"
# shellcheck disable=SC2016 # backticks are literal Slack mrkdwn, not command subst
body="$(printf 'No provider resource yet. Paths: %s\n<%s|▶ Scaffold it> · `gh workflow run scaffold-resource.yml -f family='"'"'%s'"'"'`' \
"$paths" "$ACTIONS_URL" "$tag")"
post_slack "🆕 New API family: $tag" "$body"
done < <(jq -c '.new_families[]' "$JSON")

# 2b) One message per curated NET-NEW resource inside a partial family —
# scaffoldable via stage 2, scoped to the resource name + its operations
# (the family's existing resources are untouched).
while IFS= read -r sr; do
[ -n "$sr" ] || continue
tag="$(printf '%s' "$sr" | jq -r '.tag')"
name="$(printf '%s' "$sr" | jq -r '.name')"
opids="$(printf '%s' "$sr" | jq -r '[.operations[].operation_id] | join(",")')"
oplist="$(printf '%s' "$sr" | jq -r '[.operations[] | "• `\(.method) \(.path)`"] | join("\n")')"
# shellcheck disable=SC2016 # backticks are literal Slack mrkdwn, not command subst
body="$(printf 'Net-new resource in the `%s` family (existing resources untouched):\n%s\n<%s|▶ Scaffold it> · `gh workflow run scaffold-resource.yml -f family='"'"'%s'"'"' -f resource_name='"'"'%s'"'"' -f operations='"'"'%s'"'"'`' \
"$tag" "$oplist" "$ACTIONS_URL" "$tag" "$name" "$opids")"
post_slack "🆕 New resource: $name" "$body"
done < <(jq -c '.scaffoldable_resources[]' "$JSON")

# 3) One message per partial family with UNCLAIMED ops — informational
# (stage 2 only scaffolds NEW resources, so no dispatch link).
while IFS= read -r tag; do
[ -n "$tag" ] || continue
ops="$(jq -r --arg t "$tag" '.unclaimed_operations[] | select(.tag == $t) | "• `\(.method) \(.path)` (`\(.operation_id)`)"' "$JSON")"
# shellcheck disable=SC2016 # backticks are literal Slack mrkdwn, not command subst
body="$(printf 'Modeled, but lagging these ops — extend the existing resource via a manual PR:\n%s' "$ops")"
post_slack "⚠️ Coverage gap: $tag" "$body"
done < <(jq -r '[.unclaimed_operations[].tag] | unique | .[]' "$JSON")

echo "Posted $sent Slack message(s)."
36 changes: 25 additions & 11 deletions .github/workflows/scaffold-resource.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,14 @@ on:
description: "Endpoint family tag from the drift report (e.g. AgentControl)"
required: true
type: string
resource_name:
description: "Optional: scaffold a single NET-NEW resource with this name (a scaffoldable_resources entry inside a partial family). Blank = scaffold the whole family."
required: false
type: string
operations:
description: "Optional: comma-separated operationIds the net-new resource should cover. Used with resource_name."
required: false
type: string
Comment thread
cursor[bot] marked this conversation as resolved.
notes:
description: "Optional extra instructions for the scaffolding agent"
required: false
Expand All @@ -34,7 +42,7 @@ permissions:

jobs:
scaffold:
name: Scaffold ${{ inputs.family }}
name: Scaffold ${{ inputs.resource_name || inputs.family }}
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
Expand All @@ -43,6 +51,14 @@ jobs:
run: |
echo "::error::Scaffolding agent is disabled (repository variable SCAFFOLD_AGENT_ENABLED != 'true')."
exit 1
- name: Validate scoped inputs
# Scoped mode (resource_name set) needs the operationId list — otherwise
# the agent enters SCOPED MODE with no operations and the whole family
# slice, an ambiguous instruction. Fail fast instead.
if: inputs.resource_name != '' && inputs.operations == ''
run: |
echo "::error::'operations' is required when 'resource_name' is set (scoped mode needs the comma-separated operationId list)."
exit 1
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: preview-v3 # scaffolds always target the v3 line
Expand All @@ -66,12 +82,11 @@ jobs:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ secrets.GITHUB_TOKEN }}
prompt: |
Scaffold a new resource for the LaunchDarkly Terraform provider covering
the "${{ inputs.family }}" API endpoint family.
${{ inputs.resource_name && format('SCOPED MODE — implement exactly ONE net-new resource named `{0}` for the LaunchDarkly Terraform provider, covering ONLY these operationIds: {1}. It belongs to the partial "{2}" API family; do NOT modify, regenerate, or touch the family''s EXISTING resources, and do not model operations outside that list.', inputs.resource_name, inputs.operations, inputs.family) || format('Scaffold a new resource for the LaunchDarkly Terraform provider covering the "{0}" API endpoint family.', inputs.family) }}

Inputs:
- Endpoint summary for the family: ./family-slice.json (repo root).
This is input context only — read it, but do NOT commit it.
- Endpoint summary for the family: ./family-slice.json (repo root). This
is the WHOLE family for context — read it, but do NOT commit it.${{ inputs.resource_name && format(' Model ONLY the operations listed above ({0}); ignore the rest of the family.', inputs.operations) || '' }}
- Full OpenAPI spec (for schemas): https://app.launchdarkly.com/api/v2/openapi.json
- Read .claude/skills/terraform-provider-add-resource/SKILL.md and its
references/patterns.md (both vendored into this repo), and follow that
Expand All @@ -86,15 +101,14 @@ jobs:
Additional notes from the operator: ${{ inputs.notes }}

Deliverable:
1. Create a branch named scaffold/${{ inputs.family }} (lowercased,
non-alphanumerics replaced with '-') off preview-v3.
2. Implement the resource (and data source if the family has GET-by-key),
1. Create a branch named scaffold/${{ inputs.resource_name || inputs.family }}
(lowercased, non-alphanumerics replaced with '-') off preview-v3.
2. Implement the resource (and data source if it has GET-by-key),
unit-testable helpers, acceptance tests, docs templates, and provider
registration. Update scripts/driftreport/mapping.yaml to mark the family
covered.
registration. ${{ inputs.resource_name && format('In scripts/driftreport/mapping.yaml, move these operations out of the "{0}" family''s new_resource_candidates into a new entry under its resources (the family stays partial).', inputs.family) || 'Update scripts/driftreport/mapping.yaml to mark the family covered.' }}
3. Run go build ./... and make fmt; fix what they surface.
4. Commit, push the branch, and open a DRAFT pull request against
preview-v3 titled "feat: scaffold ${{ inputs.family }} resource
preview-v3 titled "feat: scaffold ${{ inputs.resource_name || inputs.family }} resource
(autogen stage 2)". The PR body must state it is agent-scaffolded and
needs human review per stage 3 of the autogen pipeline.
Never push to preview-v3 directly and never merge anything.
Expand Down
Loading
Loading