Add Java SDK capability manifest and compatibility matrix

.pre-commit-config.yaml

@@ -280,6 +280,21 @@ repos:
           (?x)
           ^java-sdk/gradle\.properties$|
           ^java-sdk/sdk/schema/schema\.json$
+      - id: update-java-sdk-readme-matrix
+        name: Update the Java SDK compatibility matrix in java-sdk/README.md and Dokka module doc
+        entry: ./scripts/ci/prek/update_java_sdk_readme_matrix.py
+        language: python
+        files: >
+          (?x)
+          ^java-sdk/capabilities\.yaml$|
+          ^java-sdk/gradle\.properties$|
+          ^java-sdk/README\.md$|
+          ^java-sdk/sdk/module\.md$|
+          ^scripts/ci/prek/lang_sdk_compat_matrix\.py$|
+          ^scripts/ci/prek/update_java_sdk_readme_matrix\.py$
+        additional_dependencies: ['PyYAML>=6.0', 'rich>=13.6.0']
+        pass_filenames: false
+        require_serial: true
       - id: check-go-version-in-sync
         name: Check Go toolchain version is consistent across build files
         entry: ./scripts/ci/prek/check_go_version_in_sync.py

contributing-docs/30_new_language_sdk.rst

@@ -430,6 +430,11 @@ authored in the target language. An SDK declares each one independently.
     trailer (see `Native Executable Bundle Format`_), a JVM artifact embeds it in the jar,
     a Node bundle embeds it in the package.
 
+``retry-policy`` (MAY)
+    The SDK lets task code inspect a failure and override whether the task retries or fails,
+    optionally with a custom retry delay. This is distinct from reporting ``up_for_retry``:
+    an SDK can support ordinary retries without exposing a task-facing retry-policy API.
+
 ``task-state-store`` (MAY)
     The task can read and write the per-task state store.
 
@@ -491,6 +496,30 @@ native Dags they are *not applicable* (``n/a``) rather than unsupported.
     The SDK exposes an object-storage API (an ``ObjectStoragePath`` equivalent) usable from
     native Dag code.
 
+Compatibility matrix
+~~~~~~~~~~~~~~~~~~~~
+
+The dimensions above are prose; each SDK also declares them *machine-readably* in one
+hand-authored ``<sdk>/capabilities.yaml``, from which a prek hook generates its published tables:
+
+.. code-block:: text
+
+    java-sdk/capabilities.yaml          <- the only file you edit
+      |
+      |  hook: update-java-sdk-readme-matrix
+      |
+      +--> java-sdk/README.md            (contributor-facing)
+      +--> java-sdk/sdk/module.md        (Dokka -> the published API reference)
+
+The hook rewrites its targets and exits non-zero when either was stale, so a drifted table fails
+the build. Keep the manifest out of whatever the SDK publishes — it describes the SDK rather than
+being part of it; for Java that means sitting above every subproject in ``settings.gradle.kts``.
+
+To add an SDK, register it in ``LANG_SDKS`` in ``scripts/ci/prek/lang_sdk_compat_matrix.py``, write
+a ``capabilities.yaml`` in the same schema, and add the equivalent hook. Adding or renaming a
+dimension means editing ``STATE_DIMENSIONS`` / ``CAPABILITY_DIMENSIONS`` there **and** the prose
+above in the same PR — the renderer validates every manifest against that list.
+
 
 Testing
 -------

java-sdk/README.md

@@ -580,6 +580,58 @@ Close the vote, **drop** the staging repository in Nexus, remove the `dist/dev`
 candidate, fix the issue, and cut the next RC (`...-rc2`). The released version
 stays the same (e.g. `<VERSION>`); only the RC counter in the tag increments.
 
+## Compatibility matrix
+
+Which Airflow TaskInstance states and capabilities this SDK supports. This table is generated from
+[`capabilities.yaml`](capabilities.yaml); the conformance dimensions are defined in the
+[Language SDK conformance spec](https://github.com/apache/airflow/blob/main/contributing-docs/30_new_language_sdk.rst).
+Do not edit the table by hand — edit `capabilities.yaml` and let the `update-java-sdk-readme-matrix`
+prek hook regenerate it.
+
+<!-- BEGIN AUTO-GENERATED LANG-SDK COMPAT MATRIX -->
+
+*Min. Airflow version: 3.3 · supervisor schema: 2026-06-16*
+
+| Dimension | Tier | Supported | Since | Notes |
+|---|---|---|---|---|
+| **TaskInstance states** |  |  |  |  |
+| state: `success` | MUST | ✓ | 3.3 |  |
+| state: `failed` | MUST | ✓ | 3.3 |  |
+| state: `up_for_retry` | MUST | ✓ | 3.3 | RetryTask |
+| state: `skipped` | SHOULD | ✗ | – | runtime does not emit TaskState skipped yet |
+| state: `deferred` | MAY | ✗ | – | runtime does not emit DeferTask yet |
+| state: `up_for_reschedule` | MAY | ✗ | – | runtime does not emit RescheduleTask yet |
+| state: `awaiting_input` | MAY | ✗ | – | runtime does not emit AwaitInputTask yet |
+| state: `removed` | MAY | ✓ | 3.3 |  |
+| **Runtime capabilities** |  |  |  |  |
+| capability: `mixed-lang-stub-target` | MUST | ✓ | 3.3 | @task.stub |
+| capability: `task-logging` | MUST | ✓ | 3.3 | SLF4J + JPL bridged to the task log |
+| capability: `xcom-read-write` | MUST | ✓ | 3.3 |  |
+| capability: `connection-read` | MUST | ✓ | 3.3 |  |
+| capability: `variable-read-write` | MUST | ✗ | – | getVariable only; no write over the comm socket yet |
+| capability: `self-contained-bundle` | MUST | ✓ | 3.3 | Airflow metadata embedded in the jar artifact |
+| capability: `retry-policy` | MAY | ✗ | – | no task-facing retry-policy API yet |
+| capability: `task-state-store` | MAY | ✗ | – | no task-facing state-store API yet |
+| capability: `asset-state-store` | MAY | ✗ | – | no task-facing state-store API yet |
+| capability: `asset-event-emit` | MAY | ✗ | – | runtime does not emit asset events yet |
+| capability: `asset-event-read` | MAY | ✗ | – | no task-facing asset-event API yet |
+| **Native-Dag authoring** |  |  |  |  |
+| capability: `native-dag-authoring` | SHOULD | ✗ | – | native Dag authoring not implemented yet |
+| capability: `task-args` | MUST † | n/a | – |  |
+| capability: `dag-params` | MUST † | n/a | – |  |
+| capability: `taskflow-dependencies` | MUST † | n/a | – |  |
+| capability: `branching` | SHOULD † | n/a | – |  |
+| capability: `dag-test` | SHOULD † | n/a | – |  |
+| capability: `task-group` | MAY † | n/a | – |  |
+| capability: `dynamic-task-mapping` | MAY † | n/a | – |  |
+| capability: `asset-inlets-outlets` | MAY † | n/a | – |  |
+| capability: `asset-scheduling` | MAY † | n/a | – |  |
+| capability: `object-store` | MAY † | n/a | – |  |
+
+*Marks: ✓ supported · ✗ not supported · n/a not applicable. A tier marked † applies only when `native-dag-authoring` is supported.*
+
+<!-- END AUTO-GENERATED LANG-SDK COMPAT MATRIX -->
+
 ## Contributing
 
 The user implements a Java application containing task methods annotated (or

java-sdk/capabilities.yaml

@@ -0,0 +1,116 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+---
+sdk: java
+
+min_airflow_version: "3.3"
+
+# Keep in sync with airflowSupervisorSchemaVersion in gradle.properties, which is what stamps the
+# JAR manifest. The render hook fails if the two disagree.
+supervisor_schema_version: "2026-06-16"
+
+# The runtime terminates a task with SucceedTask, RetryTask, or TaskState (failed/removed); it does
+# not yet emit skipped, DeferTask, RescheduleTask, or AwaitInputTask.
+states:
+  success:
+    supported: true
+    since: "3.3"
+  failed:
+    supported: true
+    since: "3.3"
+  up_for_retry:
+    supported: true
+    since: "3.3"
+    note: "RetryTask"
+  skipped:
+    supported: false
+    note: "runtime does not emit TaskState skipped yet"
+  deferred:
+    supported: false
+    note: "runtime does not emit DeferTask yet"
+  up_for_reschedule:
+    supported: false
+    note: "runtime does not emit RescheduleTask yet"
+  awaiting_input:
+    supported: false
+    note: "runtime does not emit AwaitInputTask yet"
+  removed:
+    supported: true
+    since: "3.3"
+
+# Runtime capabilities reflect the task-facing Client surface; native-Dag authoring is not
+# implemented yet, so every native capability is unsupported.
+capabilities:
+  mixed-lang-stub-target:
+    supported: true
+    since: "3.3"
+    note: "@task.stub"
+  task-logging:
+    supported: true
+    since: "3.3"
+    note: "SLF4J + JPL bridged to the task log"
+  xcom-read-write:
+    supported: true
+    since: "3.3"
+  connection-read:
+    supported: true
+    since: "3.3"
+  variable-read-write:
+    supported: false
+    note: "getVariable only; no write over the comm socket yet"
+  self-contained-bundle:
+    supported: true
+    since: "3.3"
+    note: "Airflow metadata embedded in the jar artifact"
+  retry-policy:
+    supported: false
+    note: "no task-facing retry-policy API yet"
+  task-state-store:
+    supported: false
+    note: "no task-facing state-store API yet"
+  asset-state-store:
+    supported: false
+    note: "no task-facing state-store API yet"
+  asset-event-emit:
+    supported: false
+    note: "runtime does not emit asset events yet"
+  asset-event-read:
+    supported: false
+    note: "no task-facing asset-event API yet"
+  native-dag-authoring:
+    supported: false
+    note: "native Dag authoring not implemented yet"
+  task-args:
+    supported: false
+  dag-params:
+    supported: false
+  taskflow-dependencies:
+    supported: false
+  branching:
+    supported: false
+  dag-test:
+    supported: false
+  task-group:
+    supported: false
+  dynamic-task-mapping:
+    supported: false
+  asset-inlets-outlets:
+    supported: false
+  asset-scheduling:
+    supported: false
+  object-store:
+    supported: false

java-sdk/sdk/build.gradle.kts

@@ -265,7 +265,15 @@ sourceSets {
 
 dokka {
     moduleVersion.set(project.version.toString())
+    pluginsConfiguration.html {
+        // Widens the narrow compatibility-matrix columns; see the comments in the file.
+        customStyleSheets.from(layout.projectDirectory.file("dokka/matrix.css"))
+    }
     dokkaSourceSets.configureEach {
+        // Module-level documentation, including the generated Language SDK compatibility matrix.
+        // Dokka rejects the file unless "# Module sdk" is its very first line, so module.md carries
+        // the ASF license header just below the heading instead of above it.
+        includes.from("module.md")
         // Suppress everything in 'execution' since it's implementation detail.
         perPackageOption {
             matchingRegex = """org\.apache\.airflow\.sdk\.execution.*"""

java-sdk/sdk/dokka/matrix.css

@@ -0,0 +1,31 @@
+/*!
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/* Tier ("SHOULD †"), Supported ("n/a") and Since ("3.3") in the compatibility matrix. */
+.table--container th:nth-child(2),
+.table--container th:nth-child(3),
+.table--container td:nth-child(2),
+.table--container td:nth-child(3) {
+  min-width: 9ch;
+}
+
+.table--container th:nth-child(4),
+.table--container td:nth-child(4) {
+  min-width: 6ch;
+}

java-sdk/sdk/module.md

@@ -0,0 +1,72 @@
+# Module sdk
+
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements.  See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership.  The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License.  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied.  See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+The Apache Airflow Java SDK — author and run Airflow task implementations in JVM languages.
+
+## Language SDK compatibility matrix
+
+Which Airflow TaskInstance states and capabilities the Java SDK currently supports. The normative
+meaning of each dimension is defined in the
+[Language SDK conformance specification](https://github.com/apache/airflow/blob/main/contributing-docs/30_new_language_sdk.rst).
+
+<!-- BEGIN AUTO-GENERATED LANG-SDK COMPAT MATRIX -->
+
+*Min. Airflow version: 3.3 · supervisor schema: 2026-06-16*
+
+| Dimension | Tier | Supported | Since | Notes |
+|---|---|---|---|---|
+| **TaskInstance states** |  |  |  |  |
+| state: `success` | MUST | ✓ | 3.3 |  |
+| state: `failed` | MUST | ✓ | 3.3 |  |
+| state: `up_for_retry` | MUST | ✓ | 3.3 | RetryTask |
+| state: `skipped` | SHOULD | ✗ | – | runtime does not emit TaskState skipped yet |
+| state: `deferred` | MAY | ✗ | – | runtime does not emit DeferTask yet |
+| state: `up_for_reschedule` | MAY | ✗ | – | runtime does not emit RescheduleTask yet |
+| state: `awaiting_input` | MAY | ✗ | – | runtime does not emit AwaitInputTask yet |
+| state: `removed` | MAY | ✓ | 3.3 |  |
+| **Runtime capabilities** |  |  |  |  |
+| capability: `mixed-lang-stub-target` | MUST | ✓ | 3.3 | @task.stub |
+| capability: `task-logging` | MUST | ✓ | 3.3 | SLF4J + JPL bridged to the task log |
+| capability: `xcom-read-write` | MUST | ✓ | 3.3 |  |
+| capability: `connection-read` | MUST | ✓ | 3.3 |  |
+| capability: `variable-read-write` | MUST | ✗ | – | getVariable only; no write over the comm socket yet |
+| capability: `self-contained-bundle` | MUST | ✓ | 3.3 | Airflow metadata embedded in the jar artifact |
+| capability: `retry-policy` | MAY | ✗ | – | no task-facing retry-policy API yet |
+| capability: `task-state-store` | MAY | ✗ | – | no task-facing state-store API yet |
+| capability: `asset-state-store` | MAY | ✗ | – | no task-facing state-store API yet |
+| capability: `asset-event-emit` | MAY | ✗ | – | runtime does not emit asset events yet |
+| capability: `asset-event-read` | MAY | ✗ | – | no task-facing asset-event API yet |
+| **Native-Dag authoring** |  |  |  |  |
+| capability: `native-dag-authoring` | SHOULD | ✗ | – | native Dag authoring not implemented yet |
+| capability: `task-args` | MUST † | n/a | – |  |
+| capability: `dag-params` | MUST † | n/a | – |  |
+| capability: `taskflow-dependencies` | MUST † | n/a | – |  |
+| capability: `branching` | SHOULD † | n/a | – |  |
+| capability: `dag-test` | SHOULD † | n/a | – |  |
+| capability: `task-group` | MAY † | n/a | – |  |
+| capability: `dynamic-task-mapping` | MAY † | n/a | – |  |
+| capability: `asset-inlets-outlets` | MAY † | n/a | – |  |
+| capability: `asset-scheduling` | MAY † | n/a | – |  |
+| capability: `object-store` | MAY † | n/a | – |  |
+
+*Marks: ✓ supported · ✗ not supported · n/a not applicable. A tier marked † applies only when `native-dag-authoring` is supported.*
+
+<!-- END AUTO-GENERATED LANG-SDK COMPAT MATRIX -->