Skip to content

Bind Python temporal and UUID arguments to their native Go types - #71809

Draft
jason810496 wants to merge 4 commits into
apache:mainfrom
jason810496:feature/go-sdk/temporal-uuid-arg-binding
Draft

Bind Python temporal and UUID arguments to their native Go types#71809
jason810496 wants to merge 4 commits into
apache:mainfrom
jason810496:feature/go-sdk/temporal-uuid-arg-binding

Conversation

@jason810496

Copy link
Copy Markdown
Member

Why

Stub annotations already carry temporal and UUID formats, but Go tasks receive the values as strings and must parse them again.

What

Bind each JSON Schema format to its native Go type:

Python annotation wire format Go parameter type
datetime, pendulum.DateTime date-time time.Time
date date time.Time (midnight)
time time time.Time (zero date)
timedelta, pendulum.Duration duration time.Duration
UUID uuid uuid.UUID
// The stub is annotated (when: datetime, window: timedelta, trace_id: UUID)
// and called with the formatted strings those annotations imply.
func ViaTemporalArgs(
    ctx sdk.TIRunContext, log *slog.Logger,
    when time.Time, window time.Duration, traceID uuid.UUID,
) (any, error)
  • Convert nested slices, maps, and struct fields recursively, with path-aware decode errors.
  • Reject incompatible Go types before the task runs. Nullable values require nil-capable types.
  • Keep plain string and custom UnmarshalJSON types as escape hatches.
  • Parse fixed ISO 8601 durations and reject variable-length year or month components.

Values cross runtimes in JSON form. The example currently passes formatted strings; native Python datetime, timedelta, and UUID call arguments require the separate #71536 Core change.


Was generative AI tooling used to co-author this PR?

A Go task could only reach an upstream task's output by hand-writing a GetXCom
call against a hard-coded task id, duplicating wiring the Dag file already
owns and breaking silently whenever that upstream was renamed.

apache#69757 ships the Python half: a `@task.stub` TaskFlow call is captured at Dag
serialization as an ordered arg-binding spec and returned by ti_run. Consuming
it here lets a Go task function take the Dag's literals and upstream XComs as
ordinary typed parameters.

A function declares either flat positional parameters or a single struct whose
fields bind by name -- kwarg-style, so an unmatched field keeps its zero value
while an argument no field claims fails the task. Signature problems are caught
once at registration; per-execution arity, type and spec errors fail the task
before its body runs, replacing a silent reflect.Zero fill.
Several binding problems stayed quiet until they were expensive or confusing.
A parameter nesting an undecodable value failed on every execution rather than
once when the bundle was built. A struct carrying `arg:` tags whose single
argument no tag matched was decoded whole into the struct, so a typo'd tag
surfaced as a decode error naming the Go type instead of the argument that
matched nothing. A value_schema or from_default of the wrong wire shape was
indistinguishable from an absent one, disabling the declared-type check or
turning a captured stub default into an argument the author supposedly passed.

The XCom whole-value decode and the concurrent multi-pull failure path were
also reachable from a Dag but exercised only in their simplest shape, and the
package docs re-explained the whole binding model at three sites, burying the
rules they were meant to state.
The Edge Worker's execution API carries no argument spec at all, so failing a
task there for an argument-count mismatch blamed the Dag author for a limit of
the transport. Keeping data parameters at their Go zero values is how those
tasks behaved before binding existed, and that path is in maintenance rather
than gaining the spec.

Registration rejected struct shapes that decode without complaint -- a struct
carrying a callback alongside its data never needed the callback filled -- and
because registering a task panics, one such signature took its whole bundle
down at startup rather than the single task.

Adding a defaulted parameter to a stub is backwards compatible in Python, and
has to stay so for the Go functions already bound to that stub: the captured
default reaches the wire but needs no Go parameter to receive it.

Untagged fields matched a Go field name verbatim, which no idiomatic snake_case
stub parameter can produce, so tags were mandatory in practice and a mismatch
quietly fell back to decoding the argument whole. Folding case and underscores
makes the untagged form usable, and embedded structs now contribute their
fields the way encoding/json has all along.

A type that decodes itself from JSON also passed registration only to be
rejected at run time by a schema check judging it on its Go kind.
A Dag author who annotates a stub parameter `datetime`, `timedelta` or `UUID`
means a moment, a span or an identifier -- not the string those become on the
wire. Without this the Go task receives that string and has to parse it itself,
re-deriving a contract the annotation already stated, and getting it wrong in
ways the runtime could have caught.

The annotation is what pydantic turns into the JSON-schema `format` the
argument spec carries, so it is also what decides which Go type can receive the
value: `date-time`/`date`/`time` fill a time.Time, `duration` a time.Duration,
`uuid` a uuid.UUID. Reading it as the contract means a Go parameter the declared
Python type cannot fill now fails before the task body runs rather than binding
something the author never meant -- including a plain int declared as a
time.Duration, which would otherwise be read as a count of nanoseconds.

These values are rewritten into a spelling encoding/json can read by one walk
over the target type, so a duration is reached wherever it sits: the argument
itself, a slice element, a map value or a struct field. A single walk is what
keeps those cases consistent; reaching only the top level would leave a
`list[timedelta]` or a struct's `timedelta` field failing against a schema that
describes them perfectly well.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant