perf: dense multi-version slots for lazy addresses, lockless exec results - WIP

[datnguyencse]

Summary

Two perf changes in the parallel execution path, motivated by profiling raw-transfer blocks where post-processing of lazy addresses (beneficiary updates) dominated wall time:

1. Unified Sparse/Dense multi-version storage (mv_memory.rs)

MvMemory.data was a DashMap<_, BTreeMap<TxIdx, MemoryEntry>> for everything, plus a separate lazy_recipient_slots map for the beneficiary fast path. Replaced with a single DashMap<_, MvEntries> where:

• MvEntries::Sparse(BTreeMap) — used for storage slots, code hashes, etc. (sparse writes; needs O(log n) range queries for validation).
• MvEntries::Dense(Box<[UnsafeCell<Option<MemoryEntry>>]>) — used for lazy addresses, indexed directly by tx_idx. O(1) writes via UnsafeCell (no DashMap write lock) and cache-friendly sequential iteration for post-processing.

add_lazy_addresses atomically upgrades a location from Sparse → Dense under the DashMap shard write lock, migrating existing entries. The lazy_addresses mutex guarantees the slot is visible by the time any concurrent record() reaches the Dense fast path.

2. Lazy post-processing in one pass, parallelized state writes (pevm.rs)

Collapsed the dual fast/slow path in execute_revm_parallel's lazy post-processing into a single Dense path:

• Sequential prefix-sum computes (balance, nonce) after each tx — required because LazySender carries the nonce-check and balance-underflow checks revm skipped.
• Parallel rayon pass applies the per-tx (balance, nonce) to each tx_result.state — independent writes across tx_idx so no synchronization needed.

A Sparse fallback is kept for correctness but never executes in practice (all lazy addresses are upgraded to Dense before any read).

Which reduce the post-execution cost from 10ms to 2ms

3. Lockless execution result slots (pevm.rs)

execution_results: Vec<Mutex<Option<PevmTxExecutionResult>>> → Vec<ExecutionResultSlot> where ExecutionResultSlot wraps UnsafeCell<Option<_>>. The Block-STM scheduler already guarantees at most one writer per tx_idx, so the Mutex was pure overhead (one atomic CAS per tx on the parallel write path, one per tx on the take loop).

4. Minor

• Added as_sparse() guard in vm.rs on the three data.get().range(..).next_back() call sites; Dense entries are never reached from VM reads because is_lazy=true short-circuits with a mock account.
• Fixed bench nonce: bench_raw_transfers was using TxEnv::default() (nonce 0) against mock_account (nonce 1), causing every tx to fail nonce check.

[datnguyencse]

Gigagas Benchmark

	No. Transactions	Gas Used	Sequential (ms)	Parallel (ms)	Speedup
Raw Transfers	47,620	1,000,020,000	55.71	34.52	🟢1.61
ERC20 Transfers	37,123	1,000,019,374	164.82	54.24	🟢3.04
Uniswap Swaps	6,413	1,000,004,742	472.12	24.64	🟢19.2

ubuntu@ip-172:~/dat-pevm$ JEMALLOC_SYS_WITH_MALLOC_CONF="thp:always,metadata_thp:always" cargo bench --features global-alloc --bench gigagas
   Compiling tikv-jemalloc-sys v0.6.1+5.3.0-1-ge13ca993e8ccb9ba9847cc330696e02839f328f7
   Compiling tikv-jemallocator v0.6.1
   Compiling pevm v0.1.0 (/home/ubuntu/dat-pevm/crates/pevm)
    Finished `bench` profile [optimized] target(s) in 46.15s
     Running benches/gigagas.rs (target/release/deps/gigagas-a52aa572103a18d8)
Independent Raw Transfers/Sequential
                        time:   [55.616 ms 55.713 ms 55.814 ms]
                        change: [+0.1297% +0.3768% +0.6172%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild
Independent Raw Transfers/Parallel
                        time:   [34.311 ms 34.523 ms 34.735 ms]
                        change: [−2.2458% −1.4439% −0.6114%] (p = 0.00 < 0.05)
                        Change within noise threshold.

Independent ERC20/Sequential
                        time:   [164.62 ms 164.82 ms 165.03 ms]
                        change: [+0.0730% +0.2960% +0.5267%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
Independent ERC20/Parallel
                        time:   [53.153 ms 54.238 ms 55.268 ms]
                        change: [−5.8962% −3.7173% −1.4826%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) low mild

Independent Uniswap/Sequential
                        time:   [471.78 ms 472.12 ms 472.46 ms]
                        change: [+0.1764% +0.2905% +0.4082%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Independent Uniswap/Parallel
                        time:   [24.610 ms 24.636 ms 24.664 ms]
                        change: [−0.0135% +0.1218% +0.2731%] (p = 0.10 > 0.05)
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
  

[gemini-code-assist]

Code Review

This pull request introduces a significant performance optimization for 'lazy addresses' (hot accounts like fee recipients) by implementing a dual-layout multi-version memory structure. It adds a 'Dense' layout using boxed slices and UnsafeCell for O(1) writes and cache-friendly iteration, complementing the existing BTreeMap-based 'Sparse' layout. Additionally, the PR replaces Mutexes with lockless UnsafeCell-based slots for recording execution results and parallelizes the final state application for lazy addresses using Rayon. Feedback was provided to improve error handling consistency when loading account data from storage during the post-processing phase.

[gemini-code-assist]

The current implementation uses ok().flatten().unwrap_or_default(), which silently ignores potential storage errors. For consistency with how code_hash is handled just above (lines 287-290), it is better to propagate storage errors explicitly.

                    let (mut balance, mut nonce) = match storage.basic(&address) {
                        Ok(Some(acc)) => (acc.balance, acc.nonce),
                        Ok(None) => (U256::ZERO, 0),
                        Err(err) => return Err(PevmError::StorageError(err.to_string())),
                    };