Skip to content

feat: operation-level coverage in API drift report (autogen pipeline stage 1b)#456

Merged
monsagri merged 2 commits into
preview-v3from
v3/driftreport-v2
Jun 15, 2026
Merged

feat: operation-level coverage in API drift report (autogen pipeline stage 1b)#456
monsagri merged 2 commits into
preview-v3from
v3/driftreport-v2

Conversation

@monsagri

@monsagri monsagri commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Stage 1b of the autogen pipeline (see .claude/plans/AUTOGEN_PIPELINE.md). Extends scripts/driftreport/ from family-level to operation-level coverage for partial families.

v1 (#445) classified every spec tag as covered/triage/ignored. partial was a placeholder. v2 makes it real: a partial family must enumerate which operationIds each resource implements, and every spec op in that family must be claimed, ignored, or triaged — otherwise it's drift.

What changed

  • Schema (mapping.yaml + report.go): resources entries gain an optional {name, operations: [opId, ...]} object shape (bare string still valid = tag-level claim). New per-family ignored_operations: [{id, reason}] and triage_operations: [opId, ...].
  • Parser: specFamiliesspecOperations — tag → []SpecOperation{method, path, operationId}; key is operationId with a "METHOD path" fallback so a spec-hygiene slip surfaces as unclaimed rather than a silent skip.
  • Drift logic (new exit-2 conditions): (a) a spec op in a partial family claimed by no resource and not ignored → unclaimed; (b) a mapping operationId no longer in the spec → stale. Op-level checks are skipped when the family tag itself is stale (no double-report). New markdown sections + JSON fields for unclaimed / stale / triage ops.
  • Validation: partial families require op lists; op lists rejected on non-partial families; ignored_operations require id+reason; duplicate claims rejected.
  • Mapping re-curation:
    • AgentControl triage → partial (15 ops claimed by the 4 AI resources / 7 ignored analytics+list / 27 triage: agent-graphs, agent-optimizations, prompt-snippets, restricted-models, targeting).
    • AI Configs covered → ignored — the spec's "AI Configs" tag holds only 2 stray analytics endpoints; the CRUD surface lives under AgentControl.
    • Contexts and Metrics (beta)partial (Metrics dogfoods the post-feat: scaffold Metrics (beta) resource (autogen stage 2) #453 metric-group resource).
    • Removed stale Users (beta) tag (dropped from the spec upstream — genuine drift caught during curation).
  • Tests: 8 → 14 (unclaimed / triage / stale / ignored-suppression / stale-family-skip / ResourceEntry string|object YAML parsing).

Validation

  • go build, go vet, go test ./scripts/driftreport/...14 pass, vet clean, gofmt clean.
  • Live run against https://app.launchdarkly.com/api/v2/openapi.jsonexit 0 (no drift). 57 families; summary covered:16, ignored:19, partial:3, triage:19.

Scope / safety

  • Internal tooling only — v3 line, not customer-facing; no provider resource/schema changes.
  • Exit-code contract (0/1/2) and Slack-only output discipline unchanged.
  • No workflow changes. The drift-report.yml on main already checks out the v3 line (ref: preview-v3), so once this merges it picks up the v2 tool on the next workflow_dispatch — no PR to main needed.

Note: the repo PR template targets resource/data-source changes (acceptance tests, docs). N/A here — this is a standalone Go tool under scripts/, covered by its own unit tests.

🤖 Generated with Claude Code


Note

Low Risk
Internal scripts/driftreport tooling only; no provider resources or customer-facing behavior. CI exit-code contract is preserved with additional drift signals.

Overview
Extends scripts/driftreport (autogen pipeline stage 1b) from tag-level families to per-operationId accounting for partial families. OpenAPI parsing now builds tag → operations (method, path, operationId) via specOperations instead of path-only specFamilies.

mapping.yaml gains resource entries as bare names (covered) or {name, operations: [...]}, plus ignored_operations (id + reason) and triage_operations (informational, not drift). Validation enforces op lists only on partial families and rejects duplicate claims. buildReport adds drift for unclaimed spec ops and stale mapping opIds (skipped when the family tag itself is stale); markdown/JSON report new sections and fields. Exit code 2 still means drift; triage ops never fail the run.

Mapping re-curation: AgentControl → partial with AI resources + ignored analytics/lists + triage for agent graphs/optimizations/snippets; AI Configs → ignored (CRUD lives under AgentControl); Contexts and Metrics (beta) → partial; removed stale Users (beta). Tests grow from 8 to 14 covering op-level drift, YAML shapes, and validation edge cases.

Reviewed by Cursor Bugbot for commit 9e96230. Bugbot is set up for automated code reviews on this repo. Configure here.

…stage 1b)

Extends the drift tool from family-level to operation-level coverage for
`partial` families. v1 classified every spec tag (covered/triage/ignored);
v2 makes `partial` real — partial families must enumerate which operationIds
each resource implements, and every spec op in a partial family must be
claimed, ignored (id+reason), or triaged, else drift.

- Schema: `resources` entries gain optional `{name, operations: [opId,...]}`
  shape (bare string stays valid for tag-level claims); new per-family
  `ignored_operations: [{id, reason}]` and `triage_operations: [opId,...]`.
- Parser: `specFamilies` -> `specOperations` (tag -> []SpecOperation with
  method/path/operationId; opId fallback key "METHOD path").
- Drift logic: new exit-2 conditions — spec op in a partial family claimed by
  no resource and not ignored (unclaimed); mapping opId gone from spec (stale).
  Op-level checks are skipped when the family tag itself is stale (no
  double-report). New report sections + JSON fields for unclaimed/stale/triage.
- validate(): partial families require op lists; op lists rejected on
  non-partial families; ignored_operations require id+reason; duplicate claims
  rejected.
- Mapping re-curation: AgentControl triage->partial (15 claimed by the 4 AI
  resources / 7 ignored / 27 triage); AI Configs covered->ignored (CRUD lives
  under AgentControl); Contexts and Metrics (beta) -> partial; removed stale
  Users (beta) tag (dropped from spec upstream).
- Tests: 8 -> 14 (unclaimed/triage/stale/ignored-suppression/stale-family-skip/
  YAML string|object parsing).

Exit-code contract (0/1/2) and Slack-only output discipline unchanged; no
workflow changes. Internal tooling only — v3 line, not customer-facing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@monsagri monsagri requested a review from a team as a code owner June 15, 2026 10:27
…eader

Captures a 2026-06-15 audit of launchdarkly.com/docs (product + API) against
the live openapi.json:

- BLIND SPOTS: SSO/SAML/SCIM/domain verification and the Observability product
  config (dashboards, alerts, session replay, traces/logs) have no openapi.json
  tag, so the tool structurally cannot see them — `exit 0` does not mean those
  surfaces are covered. Documented so nobody reads a clean run as full coverage.
- PRODUCT-PILLAR -> FAMILY MAP: ties the three docs-home product brands
  (CodeControl, AgentControl, Release Management) to spec tags — notably that
  the `AgentControl` tag carries the AI Config CRUD surface (not the near-empty
  `AI Configs` tag), and that the entire Release Management pillar is still
  triage (highest-value cluster to scaffold next).

Comment-only; tool parses and exits 0 unchanged. Lives in the repo so the CI
scaffolding agent (which only sees the checkout) can read it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@monsagri monsagri merged commit 52f5ec2 into preview-v3 Jun 15, 2026
38 of 40 checks passed
@monsagri monsagri deleted the v3/driftreport-v2 branch June 15, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant