Skip to content

[All] Dynamic protobuf support: runtime descriptors and messages without codegen #99

Description

@teodordelibasic-db

Goal

Let users stream without a local .proto file or codegen step: obtain the table's protobuf descriptor at runtime, and build and ingest records dynamically against it.

Two deliverables per SDK

  • Descriptor at runtime: build a protobuf descriptor without a compiled .proto file — either from a Unity Catalog table schema, or constructed programmatically in code.
  • Dynamic messages: construct and ingest records at runtime against that descriptor, with no statically compiled proto types.

API contract

The canonical way records are supplied dynamically (native dynamic-message objects vs JSON encoded against the descriptor) is decided in the Rust ticket #372. The wrapper SDKs follow that contract, which is why they are blocked by it.

Status

Notes

Per-SDK work is tracked in the sub-issues. Supersedes #240, whose per-language descriptor work is folded into them.

Metadata

Metadata

Assignees

Labels

api-unificationCross-SDK API consistency / parityenhancementNew feature or requestfeature-requestNet-new capability requested by customers

Type

No fields configured for Task.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions