Skip to content

Conversation

@bshastry
Copy link
Contributor

PR description

This PR adds transaction tracing support to the block-test subcommand in evmtool, enabling detailed debugging of block test execution with opcode-level traces.

New Feature: -t/--trace flag

The block-test command now supports a -t or --trace flag that outputs detailed transaction execution traces in JSON format compatible with go-ethereum. This enables developers to:

  • Debug complex block test failures at the opcode level
  • Analyze gas consumption patterns
  • Inspect stack and memory state during execution
  • Compare execution behavior across different Ethereum clients

Usage

# Basic tracing
evmtool block-test -t test.json

# With stack and memory traces
evmtool block-test -t --trace-stack --trace-memory test.json

Example Trace Output

Traces are output to stderr in JSON format:

{"pc":7,"op":62,"gas":"0xf3c12b","gasCost":"0x0","memSize":0,"depth":1,"refund":0,"opName":"RETURNDATACOPY","error":"Out of bounds"}

Implementation Details

  • Integrates with existing TracerManager and StandardJsonTracer infrastructure
  • Correctly handles all transaction statuses per Ethereum protocol:
    • INVALID transactions (wrong nonce, insufficient balance) → reject block
    • FAILED transactions (REVERT, out of gas, stack underflow) → include in block with receipt
    • SUCCESSFUL transactions → include in block with receipt
  • Trace format matches go-ethereum for compatibility with existing analysis tools
  • Behavior aligns with Besu's standard block import path and reference implementations

Testing

Manually tested with fuzzer derived blocktests (about 800 of them)

Both tests include transactions with runtime errors (stack underflow, out of gas, invalid opcodes) that are correctly traced and handled.

Fixed Issue(s)

N/A - This is a new feature enhancement, not fixing a reported issue.


Thanks for sending a pull request! Have you done the following?

  • Checked out our contribution guidelines?
  • Considered documentation and added the doc-change-required label to this PR if updates are required.
  • Considered the changelog and included an update if required.
  • For database changes (e.g. KeyValueSegmentIdentifier) considered compatibility and performed forwards and backwards compatibility tests (N/A - no database changes)

Locally, you can run these tests to catch failures early:

  • spotless: ./gradlew spotlessApply
  • unit tests: ./gradlew build (evmtool has no specific unit tests for this command)
  • acceptance tests: ./gradlew acceptanceTest (not required for evmtool changes)
  • integration tests: ./gradlew integrationTest (not required for evmtool changes)
  • reference tests: ./gradlew ethereum:referenceTests:referenceTests (not applicable)
  • hive tests: Engine or other RPCs modified? (N/A - evmtool only)

Notes

  • This is a developer tooling enhancement for evmtool, not a change to core Besu functionality
  • Manually tested with complex Ethereum reference test cases
  • No changes to APIs, consensus, networking, or database
  • Documentation may be beneficial to show developers how to use the new tracing capability

@bshastry bshastry force-pushed the feature/blocktest-tracing branch from 6224044 to d35158c Compare October 15, 2025 09:38
Implements tracing functionality for the block-test subcommand in evmtool,
allowing developers to debug block test execution with detailed transaction
traces including opcodes, gas usage, stack, and memory state.

New features:
- Added -t/--trace-transactions flag to enable tracing during block test execution
- Added --trace-memory, --trace-stack, --trace-returndata, --trace-storage options
- Added --trace-output option to specify output file (default: stderr)
- Integrated with StandardJsonTracer infrastructure
- Traces are output in JSON format compatible with go-ethereum

Implementation:
The processBlockWithTracing() method processes transactions with full tracing
while maintaining correct transaction status handling. Transactions are
categorized as INVALID, FAILED, or SUCCESSFUL per Ethereum protocol:
- INVALID transactions (wrong nonce, insufficient balance) reject the block
- FAILED transactions (reverted execution) are included with receipts
- SUCCESSFUL transactions are included with receipts

This distinction ensures blocks with reverted transactions are correctly
accepted, matching behavior of the standard block import path and reference
implementations like go-ethereum.

Conflict resolution:
This commit was rebased onto main which added test summary reporting (hyperledger#9246).
Both features are now merged:
- Test summary reporting tracks pass/fail counts across all tests
- Transaction tracing provides detailed execution traces when enabled
- Both features work independently and complement each other

Usage:
  evmtool block-test -t --trace-stack --trace-memory <test-file.json>
  evmtool block-test -t --trace-output=trace.jsonl <test-file.json>

Tested with Ethereum reference test suite including complex scenarios with
failed transactions. Trace output format matches go-ethereum for compatibility
with existing tooling.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Signed-off-by: Bhargava Shastry <[email protected]>
@bshastry bshastry force-pushed the feature/blocktest-tracing branch from d35158c to bd27f72 Compare October 17, 2025 10:27
Signed-off-by: Bhargava Shastry <[email protected]>
Signed-off-by: Bhargava Shastry <[email protected]>
@lu-pinto
Copy link
Member

lu-pinto commented Nov 3, 2025

Hi @bshastry first of all, thank you for your PR. I'm wondering what's the rationale for adding tracing support to block-test? Doesn't the state-test suffice for what you need? Take a look at an example here: https://github.com/hyperledger/besu/blob/main/ethereum/evmtool/src/test/resources/org/hyperledger/besu/evmtool/state-test/clz-opcode.json This already supports tracing out-of-the-box.
What's your use case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants