feat(l1): exposing peer related metrics + panel dashboards #5520

rodrigo-o · 2025-12-04T17:10:27Z

Motivation

We want information about the current status of p2p peers in our node.

Description

This PR:

Creates a new p2p module in the metrics crate
Instruments established connections and disconnections for peers using the already available metrics.rs module in networking (which wasn't exposing metrics to prometheus but to logs instead)
Fix an issue were some disconnections weren't seen due to just reporting on the Disconnect message receive branch.
Add a new Peer Info rows and 5 panels
Updated documentation

…nnections

…er events

…ient

github-actions · 2025-12-04T17:13:41Z

Lines of code report

Total lines added: 160
Total lines removed: 0
Total lines changed: 160

Detailed view

+-----------------------------------------------------------+-------+------+
| File                                                      | Lines | Diff |
+-----------------------------------------------------------+-------+------+
| ethrex/crates/blockchain/metrics/api.rs                   | 53    | +5   |
+-----------------------------------------------------------+-------+------+
| ethrex/crates/blockchain/metrics/mod.rs                   | 50    | +2   |
+-----------------------------------------------------------+-------+------+
| ethrex/crates/blockchain/metrics/p2p.rs                   | 75    | +75  |
+-----------------------------------------------------------+-------+------+
| ethrex/crates/networking/p2p/metrics.rs                   | 619   | +20  |
+-----------------------------------------------------------+-------+------+
| ethrex/crates/networking/p2p/rlpx/connection/handshake.rs | 528   | +2   |
+-----------------------------------------------------------+-------+------+
| ethrex/crates/networking/p2p/rlpx/connection/server.rs    | 1047  | +12  |
+-----------------------------------------------------------+-------+------+
| ethrex/crates/networking/p2p/rlpx/p2p.rs                  | 361   | +44  |
+-----------------------------------------------------------+-------+------+

rodrigo-o · 2025-12-04T19:22:49Z

crates/networking/p2p/metrics.rs

        reason: DisconnectReason,
    ) {
-        self.peers.fetch_add(1, Ordering::Relaxed);
+        self.peers.fetch_sub(1, Ordering::Relaxed);


Apparently we were adding peers instead of removing them before.

rodrigo-o · 2025-12-04T20:17:27Z

crates/networking/p2p/rlpx/connection/server.rs

+                if established_state.is_validated {
+                    // If its validated the peer was connected, so we record the disconnection.
+                    let reason = established_state
+                        .disconnect_reason
+                        .unwrap_or(DisconnectReason::NetworkError);
+                    METRICS
+                        .record_new_rlpx_conn_disconnection(
+                            &established_state
+                                .node
+                                .version
+                                .clone()
+                                .unwrap_or("Unknown".to_string()),
+                            reason,
+                        )
+                        .await;
+                }


This was moved from the Disconnect message handling to the teardown to take into account other reasons for disconnection that were missing previously.

Copilot

Pull request overview

This PR adds P2P peer metrics instrumentation to expose peer connectivity data through Prometheus and Grafana dashboards. It addresses a gap in observability by tracking peer counts, client distributions, and disconnection events.

Key Changes:

Creates new p2p metrics module with gauges for peer counts and client distribution, plus counters for disconnections
Fixes a bug where disconnections weren't tracked when peers dropped without explicit Disconnect messages
Adds comprehensive Grafana dashboard panels with 5 visualizations for peer monitoring

Reviewed changes

Copilot reviewed 13 out of 20 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`crates/blockchain/metrics/p2p.rs`	New module implementing P2P metrics with peer count gauges and disconnection counters
`crates/networking/p2p/metrics.rs`	Updates metric recording logic to properly decrement peer counts and expose to Prometheus
`crates/networking/p2p/rlpx/connection/server.rs`	Adds disconnect reason tracking and validated flag to handle metrics in teardown
`crates/networking/p2p/rlpx/p2p.rs`	Adds `all()` method and test to ensure all DisconnectReason variants are tracked
`docs/developers/l1/dashboards.md`	Documents new Peer Info dashboard row with panel descriptions
`metrics/provisioning/grafana/dashboards/common_dashboards/ethrex_l1_perf.json`	Adds 5 new Grafana panels for peer visualization
`crates/blockchain/metrics/Cargo.toml`	Updates metrics feature to include prometheus dependency
`cmd/ethrex/Cargo.toml`	Enables metrics feature for ethrex-p2p crate

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

docs/developers/l1/dashboards.md

crates/networking/p2p/rlpx/p2p.rs

ElFantasma

Nice! 😎

MegaRedHand

LFG

rodrigo-o added 17 commits December 3, 2025 10:59

initial p2p metrics creation

ba1a755

modified metrics.rs in p2p to expose inc and dec of peers

8521777

missing change to cmd cargo and formatting

b02c0d4

use missing Encoder

990b80f

initial dashboard panels idea

c2a0bdc

added client_type to disconnection metrics and changed dashboards

3a2101c

some other enhancementes to the new panels

98d19f1

more panel enhancements

c606c54

move the call to conn_disconnection to the teardown of Established co…

f29c277

…nnections

remove 0 counts from the disconnection table panel

da0bdb9

adding an is_established just to test the peer drift

d633b97

Make sure we initialize metrics for reasons to avoid losing first tim…

d6dd275

…er events

set up DisconnectionReasons just the first time we encounter a new cl…

34b9b74

…ient

remove 0s from Disconection reason events

b5bd5fc

fix a plugin version issue and renamed a panel

3a939f0

avoid decimals in count panel

1b315af

Change row name and fixed an unintended filter from API errors

51ccebb

github-actions bot assigned rodrigo-o Dec 4, 2025

github-actions bot added the L1 Ethereum client label Dec 4, 2025

github-project-automation bot added this to ethrex_l1 Dec 4, 2025

rodrigo-o mentioned this pull request Dec 4, 2025

feat(l1): simple peer metrics #5502

Closed

rodrigo-o added 3 commits December 4, 2025 14:18

Add a comment related to the need for all in DisconnectReason

026fb50

add a new is_validated field to the established connection

04f7c67

comments and formatting

6c3a669

rodrigo-o commented Dec 4, 2025

View reviewed changes

rodrigo-o added 4 commits December 4, 2025 16:23

more formatting

4e9195e

updated documentation

cea214e

Merge branch 'main' into peer-panels

3e6040b

formatting

5b29365

rodrigo-o commented Dec 4, 2025

View reviewed changes

rodrigo-o marked this pull request as ready for review December 4, 2025 20:17

rodrigo-o requested a review from a team as a code owner December 4, 2025 20:17

Copilot AI review requested due to automatic review settings December 4, 2025 20:17

ethrex-project-sync bot moved this to In Review in ethrex_l1 Dec 4, 2025

Copilot started reviewing on behalf of rodrigo-o December 4, 2025 20:18 View session

Copilot finished reviewing on behalf of rodrigo-o December 4, 2025 20:19

Copilot AI reviewed Dec 4, 2025

View reviewed changes

docs/developers/l1/dashboards.md Show resolved Hide resolved

crates/networking/p2p/rlpx/p2p.rs Show resolved Hide resolved

ElFantasma approved these changes Dec 4, 2025

View reviewed changes

MegaRedHand approved these changes Dec 4, 2025

View reviewed changes

rodrigo-o added 4 commits December 4, 2025 19:05

missing imgs in docs

b125005

make client distribution instant

0d1a550

Added not about needing the consensus node running for local metrics

b0ebbce

remove old unused dashboards.yml

f22baa9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(l1): exposing peer related metrics + panel dashboards #5520

feat(l1): exposing peer related metrics + panel dashboards #5520

Uh oh!

rodrigo-o commented Dec 4, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Dec 4, 2025 •

edited

Loading

Uh oh!

rodrigo-o Dec 4, 2025

Uh oh!

rodrigo-o Dec 4, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

ElFantasma left a comment

Uh oh!

MegaRedHand left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

feat(l1): exposing peer related metrics + panel dashboards #5520

Are you sure you want to change the base?

feat(l1): exposing peer related metrics + panel dashboards #5520

Uh oh!

Conversation

rodrigo-o commented Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Lines of code report

Uh oh!

rodrigo-o Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

rodrigo-o Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

ElFantasma left a comment

Choose a reason for hiding this comment

Uh oh!

MegaRedHand left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

rodrigo-o commented Dec 4, 2025 •

edited

Loading

github-actions bot commented Dec 4, 2025 •

edited

Loading