feat: operation-level coverage in API drift report (autogen pipeline stage 1b)#456
Merged
Conversation
…stage 1b)
Extends the drift tool from family-level to operation-level coverage for
`partial` families. v1 classified every spec tag (covered/triage/ignored);
v2 makes `partial` real — partial families must enumerate which operationIds
each resource implements, and every spec op in a partial family must be
claimed, ignored (id+reason), or triaged, else drift.
- Schema: `resources` entries gain optional `{name, operations: [opId,...]}`
shape (bare string stays valid for tag-level claims); new per-family
`ignored_operations: [{id, reason}]` and `triage_operations: [opId,...]`.
- Parser: `specFamilies` -> `specOperations` (tag -> []SpecOperation with
method/path/operationId; opId fallback key "METHOD path").
- Drift logic: new exit-2 conditions — spec op in a partial family claimed by
no resource and not ignored (unclaimed); mapping opId gone from spec (stale).
Op-level checks are skipped when the family tag itself is stale (no
double-report). New report sections + JSON fields for unclaimed/stale/triage.
- validate(): partial families require op lists; op lists rejected on
non-partial families; ignored_operations require id+reason; duplicate claims
rejected.
- Mapping re-curation: AgentControl triage->partial (15 claimed by the 4 AI
resources / 7 ignored / 27 triage); AI Configs covered->ignored (CRUD lives
under AgentControl); Contexts and Metrics (beta) -> partial; removed stale
Users (beta) tag (dropped from spec upstream).
- Tests: 8 -> 14 (unclaimed/triage/stale/ignored-suppression/stale-family-skip/
YAML string|object parsing).
Exit-code contract (0/1/2) and Slack-only output discipline unchanged; no
workflow changes. Internal tooling only — v3 line, not customer-facing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eader Captures a 2026-06-15 audit of launchdarkly.com/docs (product + API) against the live openapi.json: - BLIND SPOTS: SSO/SAML/SCIM/domain verification and the Observability product config (dashboards, alerts, session replay, traces/logs) have no openapi.json tag, so the tool structurally cannot see them — `exit 0` does not mean those surfaces are covered. Documented so nobody reads a clean run as full coverage. - PRODUCT-PILLAR -> FAMILY MAP: ties the three docs-home product brands (CodeControl, AgentControl, Release Management) to spec tags — notably that the `AgentControl` tag carries the AI Config CRUD surface (not the near-empty `AI Configs` tag), and that the entire Release Management pillar is still triage (highest-value cluster to scaffold next). Comment-only; tool parses and exits 0 unchanged. Lives in the repo so the CI scaffolding agent (which only sees the checkout) can read it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stage 1b of the autogen pipeline (see
.claude/plans/AUTOGEN_PIPELINE.md). Extendsscripts/driftreport/from family-level to operation-level coverage forpartialfamilies.v1 (#445) classified every spec tag as covered/triage/ignored.
partialwas a placeholder. v2 makes it real: a partial family must enumerate whichoperationIds each resource implements, and every spec op in that family must be claimed, ignored, or triaged — otherwise it's drift.What changed
mapping.yaml+report.go):resourcesentries gain an optional{name, operations: [opId, ...]}object shape (bare string still valid = tag-level claim). New per-familyignored_operations: [{id, reason}]andtriage_operations: [opId, ...].specFamilies→specOperations— tag →[]SpecOperation{method, path, operationId}; key isoperationIdwith a"METHOD path"fallback so a spec-hygiene slip surfaces as unclaimed rather than a silent skip.operationIdno longer in the spec → stale. Op-level checks are skipped when the family tag itself is stale (no double-report). New markdown sections + JSON fields for unclaimed / stale / triage ops.ignored_operationsrequire id+reason; duplicate claims rejected.AgentControltriage → partial (15 ops claimed by the 4 AI resources / 7 ignored analytics+list / 27 triage: agent-graphs, agent-optimizations, prompt-snippets, restricted-models, targeting).AI Configscovered → ignored — the spec's "AI Configs" tag holds only 2 stray analytics endpoints; the CRUD surface lives underAgentControl.ContextsandMetrics (beta)→ partial (Metrics dogfoods the post-feat: scaffold Metrics (beta) resource (autogen stage 2) #453 metric-group resource).Users (beta)tag (dropped from the spec upstream — genuine drift caught during curation).ResourceEntrystring|object YAML parsing).Validation
go build,go vet,go test ./scripts/driftreport/...→ 14 pass, vet clean,gofmtclean.https://app.launchdarkly.com/api/v2/openapi.json→ exit 0 (no drift). 57 families; summarycovered:16, ignored:19, partial:3, triage:19.Scope / safety
drift-report.ymlonmainalready checks out the v3 line (ref: preview-v3), so once this merges it picks up the v2 tool on the nextworkflow_dispatch— no PR tomainneeded.🤖 Generated with Claude Code
Note
Low Risk
Internal
scripts/driftreporttooling only; no provider resources or customer-facing behavior. CI exit-code contract is preserved with additional drift signals.Overview
Extends
scripts/driftreport(autogen pipeline stage 1b) from tag-level families to per-operationIdaccounting forpartialfamilies. OpenAPI parsing now builds tag → operations (method,path,operationId) viaspecOperationsinstead of path-onlyspecFamilies.mapping.yamlgains resource entries as bare names (covered) or{name, operations: [...]}, plusignored_operations(id + reason) andtriage_operations(informational, not drift). Validation enforces op lists only onpartialfamilies and rejects duplicate claims.buildReportadds drift for unclaimed spec ops and stale mapping opIds (skipped when the family tag itself is stale); markdown/JSON report new sections and fields. Exit code 2 still means drift; triage ops never fail the run.Mapping re-curation:
AgentControl→ partial with AI resources + ignored analytics/lists + triage for agent graphs/optimizations/snippets;AI Configs→ ignored (CRUD lives underAgentControl);ContextsandMetrics (beta)→ partial; removed staleUsers (beta). Tests grow from 8 to 14 covering op-level drift, YAML shapes, and validation edge cases.Reviewed by Cursor Bugbot for commit 9e96230. Bugbot is set up for automated code reviews on this repo. Configure here.