[1.26.x] feat(dashboard): modernize OpenAPI SDK generation by wklken · Pull Request #2989 · TencentBlueKing/blueking-apigateway

wklken · 2026-07-10T09:15:49Z

for 1.26.x; with delete the whole esb from codebase

Description

Replace the preset SDK templates with asynchronous, database-backed OpenAPI Generator tasks for Python, Java, Go, and JavaScript (TypeScript Fetch client).

Generated packages are always stored in BKRepo Generic. Python can additionally publish to PyPI and Java to Maven when their repositories are configured. Generation and native publication have independent retries; native publication reuses the built package identity saved with the task.

A dedicated SDK worker image contains the pinned toolchains. JavaScript builds install from the checked-in dependency lock using npm ci. Django uses one set of SDK settings, and download responses share a consistent Generic artifact selection rule.

Compatibility notes

V1 generate retains HTTP 200 with data: [].
The legacy V2 generate endpoint retains HTTP 201 with data: [].
New V2 task and Web creation endpoints return HTTP 202 with a queryable task identity/status URL.
Legacy version input remains accepted; package versions derive from the resource version.
golang remains accepted at API and deployment environment boundaries and is normalized to go.
Legacy generation endpoints remain no-ops when SDK generation is disabled; Web/new task endpoints report unavailable. SDK cleanup also skips when generation is disabled.

Verification

Focused SDK/config/API suite: 408 passed.
Dashboard lint-check passed.
Pinned OpenAPI Generator -> JavaScript npm ci (TypeScript 5.9.3) -> build/package -> installed consumer passed.
Dashboard uv run make test during SDK remediation: 4231 passed (the local checkout also contained unrelated existing tests).
Final uv run make lint-check and git diff --check passed.
Frontend polling regression: 12 cases passed, including generation failure, native publication, mixed rows, and unmount.
Targeted frontend ESLint and pnpm build-only passed (existing bundle-size warnings).

The related Helm chart work is on feat/apigateway-sdk-worker-1.23 from bk-apigateway-1.23.x: https://git.woa.com/blueking/helm-charts/-/merge_requests/new?merge_request%5Bsource_branch%5D=feat%2Fapigateway-sdk-worker-1.23

Checklist

填写 PR 描述及相关 issue (write PR description and related issue)
代码风格检查通过 (code style check passed)
PR 中包含单元测试 (include unit test)
单元测试通过 (unit test passed)
本地开发联调环境验证通过 (local development environment verification passed)

Validate SDK language settings during settings construction, require Generic BKRepo configuration for generation, and enforce approved generator property keys.

Require resource-version objects as the package-version authority, freeze language properties while keeping fingerprint payloads serializable, and retain literal golang compatibility only for legacy SDK records and manager routing.

tencentblueking-adm · 2026-07-16T09:07:34Z

All committers have signed the CLA.

Why this change was needed: PR TencentBlueKing#2989 conflicted with the current master branch after the v2 sync API, configuration, and contract tests evolved. What changed: - merged upstream/master into feat/openapi-sdk-generation - preserved the asynchronous SDK generation 202 response contract - retained master Celery, configuration, API, and test changes - kept superseded legacy SDK generators and templates deleted Problem solved: The PR branch now contains the current master history with all textual and semantic conflicts resolved and verified.

Why this change was needed: The asynchronous SDK generation APIs did not expose a reliable task lifecycle, and retry, lease, legacy artifact, and native publishing edge cases could produce misleading states or unusable artifact links. What changed: - align the frontend and public APIs around observable generation tasks - harden task status, lease renewal, retry, artifact selection, and publishing recovery - synchronize OpenAPI definitions and docs with regression coverage Problem solved: SDK generation now reports terminal outcomes accurately, preserves compatibility, and recovers safely from partial publication and worker ownership changes.

…generation # Conflicts: # src/dashboard/apigateway/apigateway/tests/conf/test_utils.py

Ignore legacy SDK records in the usage-example endpoint so obsolete clients are not documented with the new OpenAPI Generator API. Render Java imports from the generated invoker package, and render Go imports with the persisted module path and package name. Add bilingual regression coverage for the corrected examples.

Why this change was needed: PR review found retry corruption, task-state races, unbounded polling, and repository credential handling gaps in the new SDK pipeline. What changed: - isolate each request and its Generic artifact ownership - guard lifecycle recovery, cleanup, and repository credentials - preserve API aliases and document the asynchronous V1 contract - bound frontend polling and handle usable partial results - add focused regression coverage for the reviewed failure paths Problem solved: SDK generation retries no longer contaminate other requests, deployment configuration failures are classified correctly, and clients receive a bounded, compatible generation flow.

Use SDK_GENERATION_ENABLED as the single source for the existing frontend SDK flags. Remove the obsolete Python SDK quota and unused log-size setting, with regression coverage for legacy environment variables.

Prevent stale item writes and inconsistent lock ordering, validate legacy SDK resource-version ownership, support native publication retries, and map legacy V2 validation failures correctly.

Use ENABLE_SDK as the only frontend SDK capability flag and remove the obsolete ALLOW_UPLOAD_SDK_TO_REPOSITORY response field, type, defaults, and consumers.

Why this change was needed: Internal superpowers design artifacts should not be included in the repository or pull request. What changed: - Removed the OAS3 resource documentation renderer design file from docs/superpowers Problem solved: The implementation remains in the branch without publishing internal planning material.

Why this change was needed: The SDK worker deployment guide should not be maintained in this repository. What changed: - Removed src/dashboard/docs/sdk-worker-deployment.md - Kept the standalone image smoke gate because it validates generated SDK artifacts across all supported languages Problem solved: The pull request no longer publishes deployment documentation while retaining useful image-level validation.

Harden the asynchronous SDK generation workflow after PR review while preserving the legacy V2 response contract. Unify language handling, centralize serialization, remove model-layer business validation, constrain subprocess execution, simplify fixed worker configuration, and keep the worker image non-root. Update focused tests and SDK smoke coverage accordingly.

Disable generated placeholder tests and share the production generator with the worker smoke check. Infer missing path parameters and avoid live backend lookups for historical SDK exports. Preserve adopted legacy SDK metadata, include the private PyPI index in installation commands, and use versioned Go module paths for major releases.

… guidance

…cies

Keep package identity and native publication choices stable across queued tasks. Accept legacy golang configuration, skip cleanup when disabled, and lock JavaScript builds with npm ci. Centralize artifact selection and retry state updates, remove redundant configuration forwarding, and apply SDK field and setting review renames.

Escape OpenAPI prose in resource docs, strip descriptions before code generation, compile Go SDKs with go build, and redact local paths from worker errors. Co-authored-by: Cursor <cursoragent@cursor.com>

Import and SDK generation now share a visited-set $ref scan, so YAML aliases cannot hang the checker and remote refs cannot reach the validator. Dashboard docs also drop <style> bodies that the XSS filter previously kept. Co-authored-by: Cursor <cursoragent@cursor.com>

wklken added 16 commits July 10, 2026 17:10

feat(sdk): define generation languages and config

4fd7656

fix(sdk): validate generation deployment config

e5a8840

Validate SDK language settings during settings construction, require Generic BKRepo configuration for generation, and enforce approved generator property keys.

fix(sdk): finalize generation config review fixes

5b02d36

Require resource-version objects as the package-version authority, freeze language properties while keeping fingerprint payloads serializable, and retain literal golang compatibility only for legacy SDK records and manager routing.

feat(sdk): persist generation tasks and artifacts

67476c9

fix(sdk): validate generation task model invariants

09ede58

fix(sdk): align migration language choices

65a3670

feat(sdk): add OpenAPI generator pipeline

ed2f187

feat(sdk): build five language artifacts

e28803c

feat(sdk): commit artifacts to BKRepo

e10330c

feat(sdk): publish native package artifacts

6821cc8

feat(sdk): orchestrate reentrant generation

797fb24

feat(sdk): run generation on dedicated queue

e60b06f

feat(sdk): expose async Web generation tasks

f54b1f3

feat(sdk): make public generation APIs async

d994d8a

refactor(sdk): replace legacy generation pipeline

216657d

build(sdk): add dedicated generation worker

d77d18a

This comment was marked as outdated.

Sign in to view

wklken marked this pull request as draft July 13, 2026 01:50

wklken added 11 commits August 31, 2026 13:12

Merge remote-tracking branch 'upstream/master' into feat/openapi-sdk-…

d5c5e58

…generation # Conflicts: # src/dashboard/apigateway/apigateway/tests/conf/test_utils.py

refactor(sdk): realign generation state model

02f4c3f

refactor(sdk): lock generation policy and fingerprint

d106dd3

refactor(sdk): generate four native client packages

359251e

refactor(sdk): make generic artifacts authoritative

910f5a4

refactor(sdk): make generation reentrant and retryable

959dffa

feat(sdk): decouple native publication status

2c3e279

wklken added 12 commits September 4, 2026 00:35

fix(sdk): close final review findings

4c1918f

refactor(sdk): unify generation feature flags

2736bc9

Use SDK_GENERATION_ENABLED as the single source for the existing frontend SDK flags. Remove the obsolete Python SDK quota and unused log-size setting, with regression coverage for legacy environment variables.

fix(sdk): harden generation retries and legacy projection

b9bb7f2

Prevent stale item writes and inconsistent lock ordering, validate legacy SDK resource-version ownership, support native publication retries, and map legacy V2 validation failures correctly.

refactor(sdk): remove legacy upload feature flag

bd71d33

Use ENABLE_SDK as the only frontend SDK capability flag and remove the obsolete ALLOW_UPLOAD_SDK_TO_REPOSITORY response field, type, defaults, and consumers.

docs(resource-doc): design OAS3 markdown renderer

862fd56

refactor(openapi): expose structured resource export

1ba8c2d

feat(resource-doc): present OAS3 operations

17c355f

feat(resource-doc): render normalized OAS3 with Jinja2

824f5e7

refactor(resource-doc): localize generation errors

7a54c2a

refactor(resource-doc): remove legacy generator dependency

ac8eece

fix(resource-doc): preserve operation display semantics

b45539c

wklken commented Sep 4, 2026

View reviewed changes

wklken added 4 commits September 4, 2026 16:41

refactor(openapi): remove unused Swagger 2 exports and refresh import…

9178125

… guidance

wklken mentioned this pull request Sep 6, 2026

[1.25.x] refactor(dashboard): migrate automatic API docs to OpenAPI 3.1 #3246

Draft

5 tasks

fix(sdk): declare consumer runtime requirements and document dependen…

16322bc

…cies

wklken commented Sep 8, 2026

View reviewed changes

Comment thread src/dashboard/apigateway/apigateway/apis/fields.py

Comment thread src/dashboard/apigateway/apigateway/conf/default.py Outdated

wklken added 4 commits September 8, 2026 11:06

fix(sdk): align Java and Go namespaces with BlueKing

4494f3a

fix(sdk): stop polling when SDK generation fails

533e0ee

refactor(sdk): remove unused code and redundant compatibility wrappers

8dc48c1

wklken changed the title ~~feat(dashboard): modernize OpenAPI SDK generation~~ [1.26.x] feat(dashboard): modernize OpenAPI SDK generation Sep 8, 2026

wklken and others added 2 commits September 8, 2026 14:48

chore(sdk): add license headers to new source files

994e3e4

fix(sdk): harden generated docs and SDK worker inputs

4c723e1

Escape OpenAPI prose in resource docs, strip descriptions before code generation, compile Go SDKs with go build, and redact local paths from worker errors. Co-authored-by: Cursor <cursoragent@cursor.com>

wklken commented Sep 8, 2026

View reviewed changes

wklken and others added 2 commits September 8, 2026 17:52

fix(sdk): preserve legacy requests and centralize generation policy

eddf504

Navigation Menu

Uh oh!

[1.26.x] feat(dashboard): modernize OpenAPI SDK generation - #2989

[1.26.x] feat(dashboard): modernize OpenAPI SDK generation#2989
wklken wants to merge 58 commits into
TencentBlueKing:masterfrom
wklken:feat/openapi-sdk-generation

wklken commented Jul 10, 2026 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Uh oh!

tencentblueking-adm commented Jul 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

wklken commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Compatibility notes

Verification

Checklist

Uh oh!

This comment was marked as outdated.

Uh oh!

tencentblueking-adm commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wklken commented Jul 10, 2026 •

edited

Loading

tencentblueking-adm commented Jul 16, 2026 •

edited

Loading