Skip to content

perf: traverse trie keys with a nibble cursor instead of materializing nibbles#647

Open
Qumeric wants to merge 1 commit into
develop-v2.1.0from
valery/mpt-nibble-cursor
Open

perf: traverse trie keys with a nibble cursor instead of materializing nibbles#647
Qumeric wants to merge 1 commit into
develop-v2.1.0from
valery/mpt-nibble-cursor

Conversation

@Qumeric

@Qumeric Qumeric commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Every get/insert/delete converted its key into a 64-nibble SmallVec up front (~64 bounds-checked pushes per operation), leaf comparison walked nibble by nibble, and new leaf paths were packed nibble by nibble — across the tens of thousands of trie operations in a block this adds up to a significant share of guest instructions.

Traversal now uses a KeyNibbles cursor: a (key bytes, nibble offset) pair that computes nibbles by shift/mask on demand.

  • No nibble array is ever materialized for the key; branch steps just advance the offset.
  • Leaf/extension path comparison exploits alignment: after a path's optional odd leading nibble, its data is whole bytes, and for whole-byte keys the cursor is byte-aligned there in every leaf comparison — so the comparison is a direct byte-slice equality check. The nibble-by-nibble fallback only runs for parity-mismatched extension paths.
  • New leaf paths built from key suffixes (encoded_path_from_key) are a one-byte prefix plus a direct copy of the key's remaining bytes, instead of a nibble-packing loop.

Node-path operations (prefix_to_nibs, split/merge path construction) are unchanged — they operate on stored paths, not lookup keys.

Benchmark results

OpenVM 2.1 / new default block

Compare-branches run vs develop-v2.1.0: 29742009056 (block 24001988, prove-stark, g7e.2xlarge):

metric develop-v2.1.0 this PR delta
execute_metered_insns 664,702,189 651,778,530 −12,923,659 (−1.94%)
total_cells 65,071,473,647 63,820,222,214 −1,251,251,433 (−1.92%)
rows 1,323,107,351 1,289,412,192 −33,695,159 (−2.55%)
app segments 73 72 −1
total_proof_time_ms 72,472 70,791 −1,681 (−2.32%)
reth-block_time_ms 388,659 387,991 −668 (−0.17%)

Previous: OpenVM 2.0 / old default block

These historical runs used the previous OpenVM 2.0 branch and old default block 23992138. They are retained for context and are not directly comparable to the OpenVM 2.1 results above.

Compare-branches run vs main (which now includes #644): 29086324164 (block 23992138, prove-stark, g7e.2xlarge):

metric main this PR delta
execute_metered_insns 708,067,128 696,408,331 −11,658,797 (−1.65%)
total_cells 64,178,350,072 63,288,917,334 −889,432,738 (−1.39%)
rows 1,319,323,277 1,305,202,103 −14,121,174 (−1.07%)
app segments 77 76 −1
total_proof_time_ms 80,021 77,778 −2,243 (−2.80%)
reth-block_time_ms 135,630 132,108 −3,522 (−2.60%)

(An earlier run against the pre-#644/#648/#649/#650 base showed −11.2M insns / −2.23% cells: 28864942969; the instruction saving carried over fully to the leaner base.)

The compare-results job failed only on the known artifact-path issue (#632); numbers were aggregated from the run artifacts directly.

@Qumeric
Qumeric requested a review from shayanh July 7, 2026 13:43
@Qumeric
Qumeric force-pushed the valery/mpt-nibble-cursor branch from 28604d5 to e8d6f42 Compare July 8, 2026 14:44
@Qumeric
Qumeric force-pushed the valery/mpt-branch-arena branch from fc5f491 to 6129977 Compare July 8, 2026 14:44
@Qumeric
Qumeric force-pushed the valery/mpt-nibble-cursor branch from e8d6f42 to 89e650d Compare July 9, 2026 12:56
@Qumeric
Qumeric force-pushed the valery/mpt-branch-arena branch 2 times, most recently from 29cc1c8 to 54edd21 Compare July 9, 2026 15:54
@Qumeric
Qumeric force-pushed the valery/mpt-nibble-cursor branch 2 times, most recently from fbf68b6 to 657bc05 Compare July 10, 2026 09:40
Base automatically changed from valery/mpt-branch-arena to main July 10, 2026 10:20
@Qumeric
Qumeric force-pushed the valery/mpt-nibble-cursor branch from 657bc05 to fb6dac3 Compare July 10, 2026 10:23
@Qumeric
Qumeric force-pushed the valery/mpt-nibble-cursor branch from fb6dac3 to 6fca225 Compare July 20, 2026 12:23
@Qumeric
Qumeric changed the base branch from main to develop-v2.1.0 July 20, 2026 12:23
@socket-security

Copy link
Copy Markdown

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action Severity Alert  (click "▶" to expand/collapse)
Warn High
Obfuscated code: cargo hyper-util is 90.0% likely obfuscated

Confidence: 0.90

Location: Package overview

From: ?cargo/reqwest@0.12.28cargo/reqwest@0.13.4cargo/hyper-util@0.1.20

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore cargo/hyper-util@0.1.20. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn High
Obfuscated code: cargo openssl is 90.0% likely obfuscated

Confidence: 0.90

Location: Package overview

From: ?cargo/reqwest@0.12.28cargo/openssl@0.10.81

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore cargo/openssl@0.10.81. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn High
Obfuscated code: cargo writeable is 90.0% likely obfuscated

Confidence: 0.90

Location: Package overview

From: ?cargo/url@2.5.8cargo/writeable@0.6.3

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore cargo/writeable@0.6.3. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn High
Obfuscated code: cargo zerocopy is 90.0% likely obfuscated

Confidence: 0.90

Location: Package overview

From: ?cargo/reqwest@0.12.28cargo/alloy-trie@0.9.4cargo/criterion@0.8.2cargo/alloy-eips@1.8.3cargo/alloy-primitives@1.6.0cargo/alloy-json-rpc@2.0.5cargo/alloy-rpc-types@2.0.5cargo/alloy-eips@2.0.5cargo/alloy-rpc-client@2.0.5cargo/alloy-consensus@2.0.5cargo/alloy-provider@2.0.5cargo/alloy@2.0.5cargo/reqwest@0.13.4cargo/revm@40.0.3cargo/reth-primitives-traits@0.4.1cargo/ark-bls12-381@0.5.0cargo/ark-bn254@0.5.0cargo/ark-ec@0.5.0cargo/ark-ff@0.5.0cargo/aurora-engine-modexp@1.2.0cargo/alloy-trie@0.8.1cargo/metrics@0.23.1cargo/revm@27.1.0cargo/bls12_381@0.8.0cargo/zerocopy@0.8.54

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore cargo/zerocopy@0.8.54. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant