Fix `BufferSource` algorithms for shared and resizable buffers by MattiasBuelens · Pull Request #1529 · whatwg/webidl

MattiasBuelens · 2025-10-08T21:34:58Z

Previously, many BufferSource algorithms were using internal slots such as [[ByteOffset]] and [[ByteLength]] directly. However, with the addition of SharedArrayBuffer (#353, #1311) and [AllowResizable] (#982), this comes with extra caveats:

To get the byte length of a SharedArrayBuffer, we need to use ArrayBufferByteLength to match SharedArrayBuffer.prototype.byteLength.
To get the byte length of a "length-tracking" typed array, we need to use TypedArrayByteLength. Similarly, for a length-tracking DataView, we need to use GetViewByteLength.
To get the byte offset of a typed array or DataView, we first need to check whether the view is still in bounds using IsArrayBufferViewOutOfBounds.

"Byte length" and "byte offset" both mirror JavaScript's asymmetry between typed arrays and DataViews: an out-of-bounds typed array reports 0, whereas an out-of-bounds DataView throws a TypeError. That way, implementations can reuse their existing getters.

When reading the length of a buffer, we use SEQ-CST ordering to match ECMAScript. When reading the contents of a buffer, we keep using UNORDERED, as "get a copy of the bytes held by the buffer source" already did. See #1529 (comment).

Transferring an ArrayBuffer must also take into account whether it should remain resizable or not. Fortunately, we can use the new ArrayBufferCopyAndDetach operation for that.

I added an optional preserveResizability parameter to "transfer an ArrayBuffer", so specifications can choose whether to preserve resizability or not. By default, this is not preserved. I intend to use it in Streams, see Add explainer for resizable buffers for BYOB readers streams#1360.
I removed the realm parameter of that algorithm, since ArrayBufferCopyAndDetach doesn't accept such a parameter. According to WebDex, no specs were actually using that parameter anyway.

Normative changes, which need a review of existing callers:

"Byte length" now throws a TypeError for a DataView that is detached or out of bounds, where it previously returned a stale [[ByteLength]]. This is reachable without [AllowResizable], simply by detaching the underlying buffer. "Get a copy of the bytes held by the buffer source" inherits this. Both algorithms have a note documenting it, but every existing caller is currently written as an infallible read.
"Transfer" no longer takes a realm parameter. Per WebDex this is not a breaking change in practice, but it does change the exported signature.
"Transfer" on a resizable ArrayBuffer now yields a fixed-length ArrayBuffer by default, instead of reusing the original buffer's resizability.

Relates to #1312.
Fixes #1385.

At least two implementers are interested (and none opposed):
- …
- …
Tests are written and can be reviewed and commented upon at:
- Wasm: test resizable/shared buffer handling in validate() and compile() web-platform-tests/wpt#62301
Implementation bugs are filed:
- Chromium: …
- Gecko: …
- WebKit: …
- Deno: …
- Node.js: …
- webidl2.js: …
- widlparser: …
MDN issue is filed: …
The top of this comment includes a clear commit message to use.

(See WHATWG Working Mode: Changes for more details.)

Preview | Diff

MattiasBuelens · 2025-11-22T20:08:11Z

@annevk Could you assign reviewer(s) to this? Thanks! 🙏

annevk · 2025-11-24T07:20:15Z

Thanks for tackling this!

Hopefully @syg and @bakkot can help or appoint someone?

annevk

I did an editorial pass.

MattiasBuelens

Quick self-review.

bakkot · 2026-01-06T00:44:37Z

+    1.  [=/Assert=]: |arrayBuffer| is an {{ArrayBuffer}} or {{SharedArrayBuffer}} object.
+    1.  Let |jsArrayBuffer| be the result of [=converted to a JavaScript value|converting=]
+        |arrayBuffer| to a JavaScript value.
    1.  If [$IsDetachedBuffer$](|jsArrayBuffer|) is true, then return the empty


If you pass a TA backed by a detached buffer, then the Set |offset| to |bufferSource|'s [=ArrayBufferView/byte offset=]. step above will throw. It's weird that passing a TA backed by a detached buffer throws while passing the detached buffer itself returns the empty byte sequence (in this step). It doesn't fundamentally break anything, but it's inconsistent.

You're right, we should check if bufferSource's underlying buffer is detached before we try to read bufferSource's byte offset.

I'll move this to the preceding "if bufferSource is a buffer view type instance" step.

I've moved the detached check all the way to the top. I also check if length is zero, so reading byte offset later on won't throw.

Funnily enough, reading bufferSource's byte length itself can still throw for an out-of-bounds DataView (as you already reported)... Do we want "get a copy of the bytes held" to propagate that error? Or should we return an empty byte sequence in that case? 🤔

I changed it so we read both byte length and byte offset before we check if length is zero. That way, any exceptions from reading those properties will be thrown first.

syg · 2026-01-15T21:43:48Z

When dealing with views backed by a SharedArrayBuffer, what memory ordering should we use? ECMAScript seems to use SEQ-CST pretty much everywhere (e.g. %TypedArray%.prototype.byteLength), but "get a copy of the bytes" uses UNORDERED for GetValueFromBuffer.

The rules of thumb here are:

Explicit accesses of the length, like via .length, or a particular builtin reading the length, of a growable SAB are seq-cst (intended to be synchronizing)
Implicit accesses of the length for the sake of bounds checking are unordered (intended to be fast)
Access of SAB (growable or otherwise) contents themselves are unordered (intended to be fast)

MattiasBuelens · 2026-08-28T09:56:59Z

@annevk @bakkot Could you re-review? Thanks. 🙏

annevk · 2026-08-28T11:46:34Z

AI found a number of substantive issues that appear to be correct: https://gist.github.com/annevk/123a3994ba82ec664b787559803cdabf (it's also incorrect about some things though; our Bikeshed setup would definitely fail on link errors).

MattiasBuelens

Quick self-review.

MattiasBuelens · 2026-08-28T19:18:39Z

-            allocated. If they are in the same [=agent cluster=], then implementations will just
-            change the backing pointers to get the same observable results with better performance
-            and no allocations.
+        *   Sufficient memory cannot be allocated in the [=current realm=], which is the [=realm=]


I'm not sure if we have to fully spell out the meaning of "current realm" here? Happy to remove that.

annevk

This looks okay to me now, but how do we want to handle tests and such?

MattiasBuelens · 2026-08-30T10:01:12Z

We'll need some APIs that already have [AllowResizable] and/or [AllowShared] attributes. Looking around a bit, I found:

WebAssembly.validate(), .compile() and .instantiate() accept [AllowResizable] AllowSharedBufferSource bytes, so that's ideal for testing the combination of both attributes. See WASM-JS-API.
Encoding is being updated to have [AllowResizable] on TextDecoder.decode() and TextEncoder.encodeInto(), see Add support for [AllowResizable] inputs encoding#362. However, that hasn't landed yet.
I'm planning to update Streams to support [AllowResizable] on ReadableStreamBYOBReader.read(view) in Add explainer for resizable buffers for BYOB readers streams#1360. (In the future, I want to also add [AllowShared].)

So I think the best way forward right now is to use the WebAssembly methods for testing. It'll be a bit awkward since the test inputs will have to be valid WASM modules, but I think I can make that work. I'll take a look. 🙂

MattiasBuelens · 2026-08-30T11:20:06Z

I've opened web-platform-tests/wpt#62301, which tests "get a copy of the bytes held by the buffer source" across all combinations of resizable/shared buffers and fixed-length/length-tracking/out-of-bounds views.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

%TypedArray%.prototype.byteOffset returns +0 for an out-of-bounds typed array; only DataView.prototype.byteOffset throws. Mirror that split, like byte length already does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Matches how IsArrayBufferViewOutOfBounds and DataView.prototype.byteLength dispatch, and removes two asserts that only restated the branch condition. Also consolidates the three per-step equivalence notes into one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Reading |view|'s byte length or byte offset throws for an out-of-bounds DataView, so the existing asserts could throw. Assert the precondition first instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Bind the resizability once instead of duplicating the ArrayBufferCopyAndDetach call, drop the redundant detached check (step 5 of that operation already throws), and use <emu-val>undefined</emu-val>. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The realm parameter was removed. ArrayBufferCopyAndDetach allocates with %ArrayBuffer%, so the new buffer is always allocated in the current realm. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Both algorithms now throw for an out-of-bounds DataView, where byte length previously returned a stale [[ByteLength]]. Call that out with algorithm-level notes, like the existing detach and transfer notes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Match the 33 existing plain Assert: steps; converting the whole file to [=/Assert=] can be a separate editorial change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Matches how BufferSource/detached is referenced throughout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

"Get a copy of the bytes" returns the empty byte sequence for a detached buffer source before it ever reads the byte length, so only a non-detached out-of-bounds DataView reaches the throwing path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This reverts commit 0c786ab.

MattiasBuelens · 2026-08-31T09:57:15Z

I ran the new tests from web-platform-tests/wpt#62301 against stable Chromium and Firefox, and there were already a few issues. 😅

Chromium currently rejects SharedArrayBuffers and DataViews on WebAssembly methods. see web-platform-tests/wpt#62301 (comment). Definitely a bug, since the Web IDL standard as published today already requires this to work. I filed #555029264 on their end.

However, when Chromium decides to fix this bug, we'll need to decide what behavior they should be implementing for DataViews, specifically for out-of-bounds DataViews.

The published Web IDL standard would read a stale [[ByteLength]] field - definitely wrong.
Gecko (in Firefox 154) treats this as if the DataView were empty, matching the behavior for an out-of-bounds typed array. This appears to be the more "natural" behavior, as browsers can treat typed arrays and DataViews more or less the same.
With this PR, browsers would need to throw instead. This aligns with ECMAScript's DataView.prototype.byteLength, but requires browsers to have separate paths for typed arrays and DataViews instead of a single ArrayBufferView path.

Note that I haven't tested WebKit yet, since I don't have a macOS device. It might be interesting to know how they handle out-of-bounds DataViews, so we can better assess the impact of this change.

@bakkot Are we still happy with this change as discussed earlier? If yes, I'll make sure to get implementer's interest from the Gecko folks, since this will change observable behavior in their implementation.

bakkot · 2026-08-31T16:04:30Z

I'll review this in a day or two though I'm somewhat rusty on the webidl side of things.

FWIW, another possible fix for the DataView issue is to make those operations not throw on DataViews in ECMAScript (and then have the web match). Making errors into non-errors is almost always web-compat. (Fun fact: ArrayBuffer.prototype.byteLength used to be spec'd to throw also; this was changed in tc39/ecma262#2164.)

MattiasBuelens · 2026-08-31T17:45:55Z

FWIW, another possible fix for the DataView issue is to make those operations not throw on DataViews in ECMAScript (and then have the web match). Making errors into non-errors is almost always web-compat. (Fun fact: ArrayBuffer.prototype.byteLength used to be spec'd to throw also; this was changed in tc39/ecma262#2164.)

Interesting! Looks like this was raised before: tc39/ecma262#2635. I'll comment there to bring this back to TC39's attention.

MattiasBuelens marked this pull request as ready for review October 13, 2025 08:14

annevk reviewed Nov 24, 2025

View reviewed changes

MattiasBuelens commented Nov 24, 2025

View reviewed changes

Comment thread index.bs Outdated

Comment thread index.bs Outdated

MattiasBuelens mentioned this pull request Jan 2, 2026

"byte length of a buffer source type" needs updating for resizable and detached buffers #1385

Open

This was referenced Jan 2, 2026

Fix IsDetachedBuffer check in ArrayBuffer transfer #1557

Open

Throwing on detached buffers is not correct jsdom/webidl-conversions#29

Open

bakkot reviewed Jan 6, 2026

View reviewed changes

Comment thread index.bs Outdated

bakkot reviewed Jan 6, 2026

View reviewed changes

Comment thread index.bs

bakkot reviewed Jan 6, 2026

View reviewed changes

MattiasBuelens force-pushed the buffer-source-fixes branch 3 times, most recently from ce8c232 to cf28af9 Compare January 11, 2026 19:00

shannonbooth mentioned this pull request Jun 21, 2026

LibWeb: Don’t crash on a detached publicExponent in generateKey LadybirdBrowser/ladybird#10215

Merged

MattiasBuelens force-pushed the buffer-source-fixes branch from cf28af9 to 6201623 Compare August 28, 2026 16:49

MattiasBuelens commented Aug 28, 2026

View reviewed changes

annevk reviewed Aug 29, 2026

View reviewed changes

Comment thread index.bs Outdated

Comment thread index.bs

annevk approved these changes Aug 30, 2026

View reviewed changes

MattiasBuelens mentioned this pull request Aug 30, 2026

Wasm: test resizable/shared buffer handling in validate() and compile() web-platform-tests/wpt#62301

Open

MattiasBuelens added 5 commits August 31, 2026 11:10

Fix buffer source's byte length

5036212

Add byte offset to ArrayBufferView

970452d

Fix writing bytes into an ArrayBufferView backed by a SharedArrayBuffer

6b6381d

Simplify writing into an ArrayBufferView

42f4781

Move assert before conversion

9837ea7

MattiasBuelens and others added 25 commits August 31, 2026 11:10

Use ArrayBufferByteLength for both ArrayBuffer and SharedArrayBuffer

bad2248

Add a "then"

c46a4be

Make byte length throw if DataView is out of bounds

1d1c894

Check if detached before reading byte offset

899c6a0

Return immediately if length is 0

28ae666

Reorder steps

ade6c84

Add "out of bounds" definition for ArrayBufferViews

dbdba4e

Read byte offset before checking length

946668e

Add notes for steps that mirror ECMAScript

c0a4fa7

Remove TODO

fbf1b09

Define "out of bounds" before its call sites

11bddb9

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Don't throw for out-of-bounds typed arrays in byte offset

4ac57f4

%TypedArray%.prototype.byteOffset returns +0 for an out-of-bounds typed array; only DataView.prototype.byteOffset throws. Mirror that split, like byte length already does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assert that view is in bounds before writing

6133e3f

Reading |view|'s byte length or byte offset throws for an out-of-bounds DataView, so the existing asserts could throw. Assert the precondition first instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Simplify ArrayBuffer/transfer

92d6c07

Bind the resizability once instead of duplicating the ArrayBufferCopyAndDetach call, drop the redundant detached check (step 5 of that operation already throws), and use <emu-val>undefined</emu-val>. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Fix dangling realm references in transfer note

ab4a930

The realm parameter was removed. ArrayBufferCopyAndDetach allocates with %ArrayBuffer%, so the new buffer is always allocated in the current realm. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Use plain "Assert:" steps

fa429eb

Match the 33 existing plain Assert: steps; converting the whole file to [=/Assert=] can be a separate editorial change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Use scoped reference for BufferSource/byte length

5a8ef66

Matches how BufferSource/detached is referenced throughout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Move and shorten note

ae01d9f

Add newline

939eb2d

Remove note since step is *actually* necessary

0c786ab

Revert "Remove note since step is *actually* necessary"

2a364f1

This reverts commit 0c786ab.

Use "the exclusive range" in note

9a418e6

MattiasBuelens force-pushed the buffer-source-fixes branch from 3fff79f to 9a418e6 Compare August 31, 2026 09:21

MattiasBuelens mentioned this pull request Aug 31, 2026

DataView byteLength/byteOffset still specified to throw when buffer detached tc39/ecma262#2635

Open

Uh oh!

Conversation

MattiasBuelens commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MattiasBuelens commented Nov 22, 2025

Uh oh!

annevk commented Nov 24, 2025

Uh oh!

annevk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MattiasBuelens left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bakkot Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

MattiasBuelens Jan 6, 2026

Choose a reason for hiding this comment

Uh oh!

MattiasBuelens Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

MattiasBuelens Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

syg commented Jan 15, 2026

Uh oh!

MattiasBuelens commented Aug 28, 2026

Uh oh!

annevk commented Aug 28, 2026

Uh oh!

MattiasBuelens left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MattiasBuelens Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

annevk left a comment

Choose a reason for hiding this comment

Uh oh!

MattiasBuelens commented Aug 30, 2026

Uh oh!

MattiasBuelens commented Aug 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MattiasBuelens commented Aug 31, 2026

Uh oh!

bakkot commented Aug 31, 2026

Uh oh!

MattiasBuelens commented Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

MattiasBuelens commented Oct 8, 2025 •

edited

Loading

MattiasBuelens left a comment •

edited

Loading

MattiasBuelens commented Aug 30, 2026 •

edited

Loading