Skip to content

Pass native datetime and UUID literals to stub tasks - #71536

Open
jason810496 wants to merge 2 commits into
apache:mainfrom
jason810496:feature/lang-sdk/stub-arg-native-values
Open

Pass native datetime and UUID literals to stub tasks#71536
jason810496 wants to merge 2 commits into
apache:mainfrom
jason810496:feature/lang-sdk/stub-arg-native-values

Conversation

@jason810496

@jason810496 jason810496 commented Aug 13, 2026

Copy link
Copy Markdown
Member

Why

A @task.stub TaskFlow call may only carry values json.dumps already knows, so the most natural arguments to pass a foreign-language task were rejected outright:

@task.stub(queue="golang")
def schedule_window(starts_at: datetime, every: timedelta, trace_id: UUID): ...


schedule_window(datetime(2024, 1, 2, 3, 4, 5), timedelta(minutes=5), UUID("6ba7b810-..."))
ValueError: @task.stub task 'schedule_window' parameter 'starts_at' received a literal of
type datetime that is not JSON-serializable, so it cannot be passed to the foreign runtime

The parameter's value_schema already told the runtime this was {"type": "string", "format": "date-time"} but the value could not travel.

What

Temporal and UUID literals are now rendered through the same Pydantic adapter that produced their value_schema, so the value and schema cannot disagree by construction. The conversion recurses through typed lists, sequences, tuples, mapping values, sets, and unions, so supported values use the same wire spelling at every depth. Sets containing supported leaves become deterministically sorted JSON arrays while uniqueItems: true tells the runtime it may reconstruct set semantics.

Python literal Wire value after this PR value_schema Rejected before? Rejected after?
datetime(2024, 1, 2, 3, 4, 5) "2024-01-02T03:04:05Z" {"type": "string", "format": "date-time"} Yes No
date(2024, 1, 2) "2024-01-02" {"type": "string", "format": "date"} Yes No
time(3, 4, 5) "03:04:05" {"type": "string", "format": "time"} Yes No
timedelta(days=1, hours=2) "P1DT2H" {"type": "string", "format": "duration"} Yes No
UUID("6BA7B810-...") "6ba7b810-..." {"type": "string", "format": "uuid"} Yes No
[datetime(2024, 1, 2, 3, 4, 5)] ["2024-01-02T03:04:05Z"] {"type": "array", "items": {"type": "string", "format": "date-time"}} Yes No
{datetime(2024, 1, 2), datetime(2024, 1, 3)} ["2024-01-02T00:00:00Z", "2024-01-03T00:00:00Z"] {"type": "array", "items": {"type": "string", "format": "date-time"}, "uniqueItems": true} Yes No
Status.READY for plain Enum rejected, no wire value {"type": "string", "enum": ["ready"], "title": "Status"} Yes Yes
Status.READY for str-backed Enum "ready" {"type": "string", "enum": ["ready"], "title": "Status"} No No
Priority.HIGH for int-backed Enum 1 {"type": "integer", "enum": [1], "title": "Priority"} No No
Decimal("1.20") rejected, no wire value number or numeric string Yes Yes
Path("/tmp/example") rejected, no wire value {"type": "string", "format": "path"} Yes Yes

Only temporal and UUID leaves opt into Pydantic serialization. Plain Enum members, Decimal, and Path retain their previous rejection behavior; string- and integer-backed Enums retain their existing JSON-native behavior, and bytes remain unsupported. Schema generation and literal conversion are deliberately separate: Pydantic may describe a type even when its Python object cannot be emitted as JSON.

Timezone-naive timestamps

A naive datetime is pinned to an explicit offset before serializing, via the same coerce_datetime the rest of Airflow uses. An offset-less timestamp is a different instant to each language runtime, so leaving it naive would make a task's behaviour depend on which language happens to run it.

Wire value Go Java Instant.parse JS new Date
2024-01-02T03:04:05Z 03:04:05Z 03:04:05Z 03:04:05Z
2024-01-02T03:04:05 03:04:05Z throws 08:04:05Z (worker-local)

Was generative AI tooling used to co-author this PR?

@jason810496 jason810496 self-assigned this Aug 13, 2026
@jason810496 jason810496 added this to the Airflow 3.4.0 milestone Aug 13, 2026
@jason810496
jason810496 force-pushed the feature/lang-sdk/stub-arg-native-values branch from da71819 to a517a7f Compare August 14, 2026 06:38
@jason810496
jason810496 marked this pull request as ready for review August 14, 2026 06:47
Comment thread airflow-core/src/airflow/serialization/stub_arg_bindings.py Outdated
Comment thread airflow-core/src/airflow/serialization/stub_arg_bindings.py
Comment thread airflow-core/src/airflow/api_fastapi/execution_api/datamodels/task_arg_binding.py Outdated
@jason810496
jason810496 marked this pull request as draft August 17, 2026 13:41
A @task.stub TaskFlow call could only carry values json.dumps already knew, so
a datetime, date, timedelta or UUID argument was rejected outright and Dag
authors had to hand-write the JSON spelling their lang SDK expected -- with
nothing keeping that spelling consistent between authors, or in step with the
value_schema the same parameter advertises.

Rendering the value through the adapter that produced its schema means the two
cannot disagree, and every language runtime sees one spelling per format.

Timestamps are pinned to an explicit offset first: an offset-less timestamp is
a different instant to each runtime -- UTC in Go, worker-local in JavaScript,
unparsable in Java -- so leaving it naive on the wire would make a task's
behaviour depend on the language that happens to run it.
@jason810496
jason810496 force-pushed the feature/lang-sdk/stub-arg-native-values branch from a517a7f to c8812b3 Compare August 19, 2026 02:48
@jason810496
jason810496 requested a lite review from Copilot August 19, 2026 02:49

This comment was marked as duplicate.

@jason810496
jason810496 requested a lite review from Copilot August 19, 2026 02:50

This comment was marked as duplicate.

@jason810496
jason810496 force-pushed the feature/lang-sdk/stub-arg-native-values branch from c8812b3 to 31e01bc Compare August 19, 2026 03:25
Pydantic JSON mode accepts a much broader set of Python objects than the temporal and UUID contract, which could silently change previously rejected values. Nested timestamps also need the same timezone normalization as top-level arguments.
@jason810496
jason810496 force-pushed the feature/lang-sdk/stub-arg-native-values branch from 31e01bc to 108dbf5 Compare August 19, 2026 04:32
@jason810496
jason810496 marked this pull request as ready for review August 19, 2026 06:05

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks TP for the review.

Comment thread airflow-core/src/airflow/serialization/stub_arg_bindings.py Outdated
Comment thread airflow-core/src/airflow/serialization/stub_arg_bindings.py
Comment thread airflow-core/src/airflow/api_fastapi/execution_api/datamodels/task_arg_binding.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants