Skip to content

Question : okio buffer for serialization formats that wrap buffers instead of creating objects #1515

Closed

Description

@balayanv

I remember years ago watching a presentation about okio buffers using and reusing segments that contain the actual byte arrays, and avoiding a whole bunch of copying + being very performant when dealing with bigger message sizes.

At the same time i want to experiment with serialization formats that directly work with bytes as in memory, instead of doing compute to create objects then doing another compute to serialize those they simply wrap a buffer, mutate directly on that buffer and when its time to write back to the wire the buffer is ready to go... with 0 cost to serialize or deserialize a message.

This poses a challenge for applications that are expecting an object and freeling passing around or holding onto those objects for some time... meaning instead of objects i have to juggle buffers/bytearrays.

I was wondering if okio buffer could do this, essentially what i need is
be able to request a mutable/resizable Buffers but when i'm done i'd call an api and let it know im done with it...

Note its very much possible that i dont even read the whole message, but few fields at specific offests, mutate it at specific offsets... maybe i wont even write some of them to the wire (e.g. the buffer is not consumed in traditional sense but i'm done with it), it can go back into the pool to be used for something else.

Activity

  1. balayanv commented on Sep 12, 2024

    @balayanv
    Author

    Another question (assuming okio buffer is viable to do this) is how does it perform with messages less than 1kb for example... lots and lots of small messages going back and forth...

  2. locked and limited conversation to collaborators on Sep 12, 2024
  3. converted this issue into a discussion #1516 on Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions