Skip to content

feat(flink-connector-ytsaurus & flink-yson): support dict and list types - #25

Open
vovapraded wants to merge 23 commits into
mainfrom
issue-24-support-dict-type
Open

feat(flink-connector-ytsaurus & flink-yson): support dict and list types#25
vovapraded wants to merge 23 commits into
mainfrom
issue-24-support-dict-type

Conversation

@vovapraded

@vovapraded vovapraded commented Mar 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds native YTsaurus dict, list, and single-level optional support for Flink MAP, ARRAY, and nullable types.

Supported

  • Reading and writing YT dicts/lists as Flink maps/arrays, including recursive nesting.
  • Non-consecutive optional<T> at any level: columns, list items, and dict values.
  • On write, null is allowed only for optional YT types; on read, # is allowed only for nullable Flink types.
  • Native scalar conversion inside nested types and validation of malformed dicts.

Limitations

  • optional<optional<T>> is rejected because Flink nullability cannot represent all three YT states.
  • Dict keys must be strings.
  • Duplicate dict keys and entry order are not preserved.

@vovapraded
vovapraded force-pushed the issue-24-support-dict-type branch 5 times, most recently from 5c80ddc to 5baec08 Compare March 16, 2026 09:52
@TheSeems
TheSeems self-requested a review March 31, 2026 14:36

@TheSeems TheSeems left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The base solution looks good. Though we need more validation and account for nested maps/lists/arrays

@vovapraded vovapraded changed the title support dict type feat(flink-connector-ytsaurus & flink-yson): support dict type Apr 30, 2026
@vovapraded
vovapraded force-pushed the issue-24-support-dict-type branch from 6912cb5 to 141b8e2 Compare July 16, 2026 09:39
@TheSeems
TheSeems self-requested a review July 24, 2026 13:18
@vovapraded vovapraded changed the title feat(flink-connector-ytsaurus & flink-yson): support dict type feat(flink-connector-ytsaurus & flink-yson): support dict and list types Aug 6, 2026
@vovapraded
vovapraded force-pushed the issue-24-support-dict-type branch from 63c70c0 to eb0aef7 Compare August 11, 2026 14:48
@vovapraded
vovapraded force-pushed the issue-24-support-dict-type branch from eb0aef7 to b8fe21c Compare August 11, 2026 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants