Skip to content

Make message.to_dict() return bytes for DATA type field - #386

Merged
haata merged 4 commits into
capnproto:masterfrom
BrianXu0623:brianxu/data_type_memory_view_fix
Sep 30, 2025
Merged

Make message.to_dict() return bytes for DATA type field#386
haata merged 4 commits into
capnproto:masterfrom
BrianXu0623:brianxu/data_type_memory_view_fix

Conversation

@BrianXu0623

@BrianXu0623 BrianXu0623 commented Sep 30, 2025

Copy link
Copy Markdown
Contributor

This PR is for resolving the issue Regression in 2.2.0: yaml.dump can no longer save capnp messages with Data type fields #385

And also trying to resolve the issue PyPI 2.2.0 source distribution missing PyCustomMessageBuilder.h #384

Background

Cap’n Proto’s C++ layer exposes Data fields as pointers into the message’s memory. Returning bytes forces an immediate copy of that buffer. To avoid that expensive copy (which is a real performance and memory bottleneck for large messages), I made it return a memoryview instead of bytes.

However, memoryview is subject to the limitation that they cannot be pickled, but in C++ this is unnecessary.

This can be resolved by modifying the message.to_dict() method, without needing change to user's code.

Please note that the changes in the test case are simply a rollback, not the introduction of any new modifications.

For the manylinux2014 issue, I added the py_custom_message_builder.h to the binary output. Hopefully it can resolve the issue, but I cannot find a way to validate it.

@BrianXu0623 BrianXu0623 changed the title change to_dict return bytes for DATA type change message.to_dict() return bytes for DATA type field Sep 30, 2025
@BrianXu0623 BrianXu0623 changed the title change message.to_dict() return bytes for DATA type field Make message.to_dict() return bytes for DATA type field Sep 30, 2025
@haata
haata merged commit c14fd70 into capnproto:master Sep 30, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants