Skip to content

[Python] Optimize Prepend: handle alignment and byte writes in one pass #8809

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
satur9nine wants to merge 3 commits into from
Closed

[Python] Optimize Prepend: handle alignment and byte writes in one pass #8809

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f2e6f61
Optimize Offset/Pad/Prep: use cached head and slicing, reduce casting
satur9nine Nov 27, 2025
9042940
Optimize Prepend: handle alignment and byte writes in one pass
satur9nine Nov 27, 2025
3080feb
Merge branch 'master' into python-perf-prepend
jtdavis777 Dec 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 25 additions & 3 deletions python/flatbuffers/builder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
from . import packer
from .compat import memoryview_type
from .compat import NumpyRequiredForThisFeature, import_numpy
from .compat import range_func
from .number_types import (SOffsetTFlags, UOffsetTFlags, VOffsetTFlags)

np = import_numpy()
Expand Down Expand Up @@ -640,8 +639,31 @@ def FinishSizePrefixed(self, rootTable, file_identifier=None):

## @cond FLATBUFFERS_INTERNAL
def Prepend(self, flags, off):
self.Prep(flags.bytewidth, 0)
self.Place(off, flags)
size = flags.bytewidth
Copy link
Contributor

@fliiiix fliiiix Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this primary faster because it skips self.Pad(alignSize) ? I guess this is a value judgment if we want more performance or simpler code here - im fine with both

Copy link
Contributor Author

@satur9nine satur9nine Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes exactly, calling Pad is what is making this original implementation expensive, inlining the implementations and avoiding Pad helps

Copy link
Contributor Author

@satur9nine satur9nine Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After I improved #8808 I see this change really gives a very small bump for all that inlining, feel free to take it or drop it, the existing code is definitely more readable and most of the gains come from part 1 and 2.

Copy link
Collaborator

@jtdavis777 jtdavis777 Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given this is such a small improvement I'm tempted to drop it -- especially with no commentary :)

Copy link
Contributor Author

@satur9nine satur9nine Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, most improvements come from the previous commits!

if size > self.minalign:
self.minalign = size

head = self.head
buf_len = len(self.Bytes)
alignSize = (~(buf_len - head)) + 1
alignSize &= size - 1

needed = alignSize + size
while head < needed:
oldBufSize = buf_len
self.GrowByteBuffer()
buf_len = len(self.Bytes)
head += buf_len - oldBufSize

if alignSize:
new_head = head - alignSize
self.Bytes[new_head:head] = b"\x00" * alignSize
head = new_head

N.enforce_number(off, flags)
head -= size
self.head = head
encode.Write(flags.packer_type, self.Bytes, head, off)

def PrependSlot(self, flags, o, x, d):
if x is not None:
Expand Down