Skip to content

fix(zkvm): provide single-threaded __atomic_* builtins for the rv64im guest - #106

Closed
Gabriel-Trintinalia wants to merge 1 commit into
Consensys-Incorporated:mainfrom
Gabriel-Trintinalia:fix/guest-atomic-builtins
Closed

fix(zkvm): provide single-threaded __atomic_* builtins for the rv64im guest#106
Gabriel-Trintinalia wants to merge 1 commit into
Consensys-Incorporated:mainfrom
Gabriel-Trintinalia:fix/guest-atomic-builtins

Conversation

@Gabriel-Trintinalia

Copy link
Copy Markdown
Collaborator

Problem

Since d399be9 (#97), every real mainnet block crashes in the ZisK guest:

thread 'main' panicked at zisk/core/src/mem.rs:669:
Mem::write_silent() invalid addr=2684354552=9ffffff8 write section start=a0000000 end=c0000000

0x9ffffff8 is 8 bytes below the floor of the entire writable region — sp has descended the full 4 MiB stack. All 500 blocks of the mainnet corpus fail; the Amsterdam synthetic fixtures and every native suite (zig build test, blockchain-tests, zkevm) pass, which is why it merged. It also blocks zesu-zkvm from pinning ZESU_REF past 65f2ca8.

Root cause — not #97

compiler_rt's __atomic_<op>_N picks its implementation on @sizeOf(T) > largest_atomic_size:

inline fn atomic_load_N(comptime T: type, src: *T, model: i32) T {
    if (@sizeOf(T) > largest_atomic_size) { ...spinlock... }
    else return @atomicLoad(T, src, .seq_cst);
}

largest_atomic_size is derived from the pointer width (8 on rv64), not from whether the target actually has atomic instructions. Our guest target does cpu_features_sub = .{ .a, .zaamo, .zalrsc } (ZisK's ISA is rv64im), so a u64 takes the "native atomic" branch, LLVM cannot lower it, and emits a libcall back into the very same function.

The disassembly is an unconditional self-call with no base case:

0000000080139c7c <compiler_rt.atomics.__atomic_load_8>:
  80139c7c: addi  sp, sp, -0x10
  80139c7e: sd    ra, 0x8(sp)
  80139c80: sd    s0, 0x0(sp)
  80139c82: addi  s0, sp, 0x10
  80139c84: li    a1, 0x5
  80139c86: auipc ra, 0x0 ; jalr -0xa(ra)   → 0x80139c7c

16 bytes of stack per turn exhausts 4 MiB in ~262k frames. Confirmed from an emulator change-trace of the final steps before the fault.

And the callers:

heap.ArenaAllocator.alloc   → __atomic_load_8, __atomic_store_8, __atomic_fetch_add_8,
                               __atomic_fetch_or_8, __atomic_exchange_8, __atomic_compare_exchange_8
heap.ArenaAllocator.free    → __atomic_load_8, __atomic_compare_exchange_8
heap.ArenaAllocator.resize  → __atomic_load_8, __atomic_compare_exchange_8
heap.ArenaAllocator.remap   → __atomic_load_8, __atomic_compare_exchange_8

#97 is not at fault and needs no change. std.heap.ArenaAllocator was simply the first guest code to reach an atomic. Any other caller — a refcount, a std container that grew one internally, a future std.once — would have hit the identical wall, only on real blocks and never in the native suites.

Change

src/zkvm/atomics.zig defines the builtins for widths 1/2/4/8 across load / store / exchange / compare_exchange / fetch_{add,sub,and,or,xor,nand}. The linker resolves them from zesu.o and never pulls compiler_rt's. Plain loads and stores are correct because the guest is strictly single-threaded — one hart, no interrupts, no preemption — so no operation here can be observed partially; the memory-order argument is ignored.

The _16 variants are deliberately not defined: 16 > largest_atomic_size, so those already take compiler_rt's spinlock path and do not recurse.

Verification

All at main (99d2546) with #97's arenas untouched:

  • mainnet_fusaka_24758573 (zesu-zkvm vector) reproduces its expected root under ziskemu 1.1.0-alpha — the same vector panics without this change
  • zig build test passes
  • 500-block mainnet corpus: 500 blocks, zero errors

Trace cells over the corpus: 12,040,230,859,277 vs 11,960,081,652,573 at 65f2ca8 (+0.67%) — that delta is main's accumulated changes since 65f2ca8, not a cost of this fix.

Repro (~30s)

cd <zesu@99d2546> && zig build rv64im-object -Doptimize=ReleaseFast
cd <zesu-zkvm>/zisk && zig build -Doptimize=ReleaseFast -Dzesu_obj=<...>/zig-out/lib/zesu.o
ziskemu -e zig-out/bin/zesu-zisk -i <zesu-zkvm>/vectors/mainnet_fusaka_24758573.bin -o /tmp/out

Follow-up

Unblocks bumping ZESU_REF in zesu-zkvm to zesu main; it has been pinned to 65f2ca8 since Aug 31 solely because of this.

🤖 Generated with Claude Code

… guest

Any guest code that performs an atomic operation blows the 4 MiB stack on
the first call, and dies as a write below the stack region:

    Mem::write_silent() invalid addr=9ffffff8 write section start=a0000000

compiler_rt's __atomic_<op>_N chooses between a spinlock and a native
atomic on `@sizeOf(T) > largest_atomic_size`, and largest_atomic_size is
derived from the pointer width (8 on rv64) rather than from whether the
target has atomic instructions. Our guest subtracts .a/.zaamo/.zalrsc
because ZisK's ISA is rv64im, so a u64 takes the "native" branch, LLVM
cannot lower it, and it emits a libcall straight back into the same
function. The disassembly is an unconditional self-call:

    80139c7c <compiler_rt.atomics.__atomic_load_8>:
      80139c84: li    a1, 0x5
      80139c86: auipc ra, 0x0; jalr -0xa(ra)   -> 80139c7c

16 bytes of stack per turn exhausts 4 MiB in ~262k frames.

Define the builtins here instead. The linker then resolves them from
zesu.o and never pulls compiler_rt's versions. Plain loads and stores are
correct because the guest is strictly single-threaded — one hart, no
interrupts, no preemption — so nothing can observe a partial operation.

This surfaced as "mainnet blocks crash in the guest since Consensys-Incorporated#97", because
std.heap.ArenaAllocator was the first code to reach an atomic. Consensys-Incorporated#97 is not
at fault and needs no change; every other caller of an atomic would have
hit the same wall, only ever on real blocks and never in the native
suites.

Verified at main (99d2546) with Consensys-Incorporated#97's arenas untouched: the
mainnet_fusaka_24758573 vector reproduces its expected root under
ziskemu 1.1.0-alpha, `zig build test` passes, and the 500-block mainnet
corpus completes with zero errors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Gabriel-Trintinalia

Copy link
Copy Markdown
Collaborator Author

Closed in favour of #107

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant