Bind Python temporal and UUID arguments to their native Go types - #71809
Draft
jason810496 wants to merge 4 commits into
Draft
Bind Python temporal and UUID arguments to their native Go types#71809jason810496 wants to merge 4 commits into
jason810496 wants to merge 4 commits into
Conversation
A Go task could only reach an upstream task's output by hand-writing a GetXCom call against a hard-coded task id, duplicating wiring the Dag file already owns and breaking silently whenever that upstream was renamed. apache#69757 ships the Python half: a `@task.stub` TaskFlow call is captured at Dag serialization as an ordered arg-binding spec and returned by ti_run. Consuming it here lets a Go task function take the Dag's literals and upstream XComs as ordinary typed parameters. A function declares either flat positional parameters or a single struct whose fields bind by name -- kwarg-style, so an unmatched field keeps its zero value while an argument no field claims fails the task. Signature problems are caught once at registration; per-execution arity, type and spec errors fail the task before its body runs, replacing a silent reflect.Zero fill.
Several binding problems stayed quiet until they were expensive or confusing. A parameter nesting an undecodable value failed on every execution rather than once when the bundle was built. A struct carrying `arg:` tags whose single argument no tag matched was decoded whole into the struct, so a typo'd tag surfaced as a decode error naming the Go type instead of the argument that matched nothing. A value_schema or from_default of the wrong wire shape was indistinguishable from an absent one, disabling the declared-type check or turning a captured stub default into an argument the author supposedly passed. The XCom whole-value decode and the concurrent multi-pull failure path were also reachable from a Dag but exercised only in their simplest shape, and the package docs re-explained the whole binding model at three sites, burying the rules they were meant to state.
The Edge Worker's execution API carries no argument spec at all, so failing a task there for an argument-count mismatch blamed the Dag author for a limit of the transport. Keeping data parameters at their Go zero values is how those tasks behaved before binding existed, and that path is in maintenance rather than gaining the spec. Registration rejected struct shapes that decode without complaint -- a struct carrying a callback alongside its data never needed the callback filled -- and because registering a task panics, one such signature took its whole bundle down at startup rather than the single task. Adding a defaulted parameter to a stub is backwards compatible in Python, and has to stay so for the Go functions already bound to that stub: the captured default reaches the wire but needs no Go parameter to receive it. Untagged fields matched a Go field name verbatim, which no idiomatic snake_case stub parameter can produce, so tags were mandatory in practice and a mismatch quietly fell back to decoding the argument whole. Folding case and underscores makes the untagged form usable, and embedded structs now contribute their fields the way encoding/json has all along. A type that decodes itself from JSON also passed registration only to be rejected at run time by a schema check judging it on its Go kind.
A Dag author who annotates a stub parameter `datetime`, `timedelta` or `UUID` means a moment, a span or an identifier -- not the string those become on the wire. Without this the Go task receives that string and has to parse it itself, re-deriving a contract the annotation already stated, and getting it wrong in ways the runtime could have caught. The annotation is what pydantic turns into the JSON-schema `format` the argument spec carries, so it is also what decides which Go type can receive the value: `date-time`/`date`/`time` fill a time.Time, `duration` a time.Duration, `uuid` a uuid.UUID. Reading it as the contract means a Go parameter the declared Python type cannot fill now fails before the task body runs rather than binding something the author never meant -- including a plain int declared as a time.Duration, which would otherwise be read as a count of nanoseconds. These values are rewritten into a spelling encoding/json can read by one walk over the target type, so a duration is reached wherever it sits: the argument itself, a slice element, a map value or a struct field. A single walk is what keeps those cases consistent; reaching only the top level would leave a `list[timedelta]` or a struct's `timedelta` field failing against a schema that describes them perfectly well.
This was referenced Aug 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
jason810496/airflow@feature/go-sdk/taskflow-arg-binding...feature/go-sdk/temporal-uuid-arg-binding
Why
Stub annotations already carry temporal and UUID formats, but Go tasks receive the values as strings and must parse them again.
What
Bind each JSON Schema
formatto its native Go type:formatdatetime,pendulum.DateTimedate-timetime.Timedatedatetime.Time(midnight)timetimetime.Time(zero date)timedelta,pendulum.Durationdurationtime.DurationUUIDuuiduuid.UUIDstringand customUnmarshalJSONtypes as escape hatches.Values cross runtimes in JSON form. The example currently passes formatted strings; native Python
datetime,timedelta, andUUIDcall arguments require the separate #71536 Core change.Was generative AI tooling used to co-author this PR?