Skip to content

Add Java SDK capability manifest and compatibility matrix - #71151

Merged
jason810496 merged 2 commits into
apache:mainfrom
jason810496:ci/java-sdk/compat-matrix
Aug 20, 2026
Merged

Add Java SDK capability manifest and compatibility matrix#71151
jason810496 merged 2 commits into
apache:mainfrom
jason810496:ci/java-sdk/compat-matrix

Conversation

@jason810496

@jason810496 jason810496 commented Aug 5, 2026

Copy link
Copy Markdown
Member

Why

The Language SDK conformance spec (#70717) defines which TaskInstance states and capabilities a Language SDK may declare, but nothing lets an SDK say what it actually supports.

This makes that support machine-readable and generates the docs from it.

How

java-sdk/capabilities.yaml          <- the only file you edit
  |
  |  render: scripts/ci/prek/lang_sdk_compat_matrix.py
  |  hook: update-java-sdk-readme-matrix
  |
  +--> java-sdk/README.md            (contributor-facing)
  |
  +--> java-sdk/sdk/module.md        (Dokka module doc -> the published API reference)

Rejected Alternative

Verfication

Screenshot 2026-08-05 at 1 24 37 PM
Was generative AI tooling used to co-author this PR?

Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/conformance/Capabilities.kt Outdated
Comment thread java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/conformance/Capabilities.kt Outdated
Comment thread .pre-commit-config.yaml Outdated
Comment thread .pre-commit-config.yaml Outdated
@jason810496
jason810496 marked this pull request as draft August 8, 2026 07:20
@jason810496
jason810496 force-pushed the ci/java-sdk/compat-matrix branch 2 times, most recently from 32576e5 to a8e0e13 Compare August 10, 2026 02:50
@jason810496
jason810496 marked this pull request as ready for review August 10, 2026 03:19
The Language SDK conformance spec defines which TaskInstance states and
capabilities a Language SDK may declare, but nothing lets an SDK say what it
actually supports. Readers of the Java SDK docs cannot tell which parts of the
spec the runtime implements today — and the gaps are real and moving (native Dag
authoring, deferral, and the state stores are not there yet), so an unqualified
"see the conformance spec" overstates what a Java task can do.

The manifest is hand-authored YAML rather than something derived from the SDK's
own sources: it describes the SDK instead of being part of it, so it has no
business shipping in a user's runtime dependencies, and deriving it would mean a
JDK and a Gradle run in a hook that only ever renders a table. It therefore sits
above every subproject in settings.gradle.kts, where no source set can pick it
up. Validation carries the weight a compiler would have: unknown keys are
rejected along with missing ones, since it is the only thing standing between a
typo and a wrong published table.

A prek hook regenerates the contributor-facing README table and the Dokka module
doc from the manifest, so a capability landing in the runtime is a one-line edit
rather than three tables to update by hand. The shared schema, SDK registry, and
renderer let the Go and TypeScript SDKs declare theirs the same way.
@jason810496
jason810496 force-pushed the ci/java-sdk/compat-matrix branch from a8e0e13 to c34b93a Compare August 16, 2026 08:14
@jason810496
jason810496 requested review from phanikumv and uranusjr and removed request for uranusjr August 17, 2026 01:51
Comment thread scripts/ci/prek/lang_sdk_compat_matrix.py
Comment thread contributing-docs/30_new_language_sdk.rst Outdated
Comment thread java-sdk/sdk/dokka/matrix.css Outdated
@jason810496
jason810496 marked this pull request as draft August 17, 2026 08:20
Contradictory native-Dag declarations could pass validation while being rendered as not applicable. Published matrix tables also need to remain readable without forcing unrelated headers onto one line.
@jason810496
jason810496 marked this pull request as ready for review August 18, 2026 08:09

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks TP for the review.

Comment thread scripts/ci/prek/lang_sdk_compat_matrix.py
Comment thread java-sdk/sdk/dokka/matrix.css Outdated
Comment thread contributing-docs/30_new_language_sdk.rst Outdated
@jason810496
jason810496 requested a review from uranusjr August 20, 2026 05:39
@jason810496
jason810496 merged commit b304b45 into apache:main Aug 20, 2026
160 of 170 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants