Skip to content

Comments

Optimize dynamic workflow by pruning node inputs (#6888)#6908

Open
VishnuVV27 wants to merge 1 commit intoflyteorg:masterfrom
VishnuVV27:fix-dynamic-workflow-sluggishness
Open

Optimize dynamic workflow by pruning node inputs (#6888)#6908
VishnuVV27 wants to merge 1 commit intoflyteorg:masterfrom
VishnuVV27:fix-dynamic-workflow-sluggishness

Conversation

@VishnuVV27
Copy link

Tracking issue

Closes #6888

Why are the changes needed?

Dynamic workflows in Flyte can fail or suffer from performance issues when the dynamic closure becomes too large, exceeding gRPC message limits (the "message too large" error). Even if execution succeeds, the Flyte UI becomes sluggish because it has to download and parse these massive blobs. This is primarily caused by the inclusion of all node inputs in the dynamic closure, which is redundant since this data is already persisted in metadata storage.

What changes were proposed in this pull request?

  1. Pruning Logic: Added a clearNodeInputs helper in flytepropeller/pkg/controller/nodes/dynamic/dynamic_workflow.go to nullify the Inputs field for nodes in the dynamic closure.
  2. Recursive Support: The function specifically handles ArrayNode (Map Tasks) to ensure nested node inputs are also pruned, which are often the largest contributors to payload bloat.
  3. Integration: Integrated the pruning step into buildContextualDynamicWorkflow so it triggers just before the closure is cached and sent as an event.

This effectively reduces the "blueprint" size of the dynamic workflow without affecting execution, as nodes continue to read their actual inputs from the defined storage locations.

How was this patch tested?

Added a new test suite TestClearNodeInputs_WithCoreBindings in flytepropeller/pkg/controller/nodes/dynamic/dynamic_workflow_test.go.

  • Positive Case: Verified that standard node inputs are set to nil.
  • Nested Case: Verified that inputs inside ArrayNode targets are also nullified.
  • Verification Command:
    go test -v ./pkg/controller/nodes/dynamic/... -run TestClearNodeInputs_WithCoreBindings

Signed-off-by: vishnuvv27 <vardhanvishnu520@gmail.com>
@VishnuVV27 VishnuVV27 force-pushed the fix-dynamic-workflow-sluggishness branch from 60404f5 to 5da02b4 Compare February 9, 2026 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Core feature] Reduce load from dynamic closure

1 participant