ARC testnet consensus not syncing

[cl-co-dr]

Arc testnet consensus node is not syncing after initiating with snapshot.
attaching error logs

Apr 11 19:13:20 arc-con[1162530]: 2026-04-11T17:13:20.841592Z INFO node{moniker=zesty-ranger-297}:consensus{height=36633465 round=0}: Scheduling timeout round=0 step=Propose Apr 11 19:13:20 arc-con[1162530]: 2026-04-11T17:13:20.841609Z INFO node{moniker=zesty-ranger-297}:consensus{height=36633465 round=0}: Phase transition prev=Read y new=Running Apr 11 19:13:20 arc-con[1162530]: 2026-04-11T17:13:20.879444Z WARN WebSocket connection failed, retrying in 7.404874563s ws_url=wss://rpc.blockdaemon.testnet.arc.network/ error=HTTP error: 405 Method Not Allowed attempt=1 Apr 11 19:13:20 arc-con[1162530]: 2026-04-11T17:13:20.922005Z INFO Connected to WebSocket ws_url=wss://rpc.quicknode.testnet.arc.network/ Apr 11 19:13:20 arc-con[1162530]: 2026-04-11T17:13:20.922901Z ERROR Received close frame with data frame=Some(CloseFrame { code: Away, reason: Utf8Bytes(b"upstream went away") }) Apr 11 19:13:20 arc-con[1162530]: 2026-04-11T17:13:20.970343Z INFO Connected to WebSocket ws_url=wss://rpc.drpc.testnet.arc.network/ Apr 11 19:13:20 arc-con[1162530]: 2026-04-11T17:13:20.974960Z ERROR Received close frame with data frame=Some(CloseFrame { code: Away, reason: Utf8Bytes(b"upstream went away") }) Apr 11 19:13:20 arc-con[1162530]: 2026-04-11T17:13:20.985660Z INFO Subscribed to newHeads ws_url=wss://rpc.drpc.testnet.arc.network/ Apr 11 19:13:21 arc-con[1162530]: 2026-04-11T17:13:21.006798Z INFO node{moniker=zesty-ranger-297}:sync{tip_height=36633464 sync_height=36633465}: SYNC REQUIRED: Falling behind tip_height=36633464 sync_height=36633465 peer_height=36641617

arc_consensus.log

[Bela1401]

#20

[cl-co-dr]

I have tried this start execution with full and then reverting from full to normal but still facing the issue . Tried different snapshots as well

[cl-co-dr]

@jbell-circle metric below
malachitebft_core_consensus_connected_peers{moniker="happy-ranger-531"} 0
confirms that no peer

[Bela1401]

I don’t think this is the same root cause as #20.

Issue #20 was the EL bootstrap inconsistency from the older pruned snapshot. That one was resolved by starting the execution layer once with --full, and you already confirmed you tried that plus fresh snapshots.

One important detail from the current docs: the recommended CL startup for a regular node uses --follow, and in follow mode the node syncs from trusted RPC endpoints rather than participating in P2P consensus. Because of that, malachitebft_core_consensus_connected_peers = 0 does not by itself prove that the root cause is missing peers.

Also, the 405 Method Not Allowed against the Blockdaemon websocket looks expected. The current run guide explicitly notes that the Blockdaemon endpoint does not currently support WebSocket connections, while HTTP fetching still works.

So I would focus the next round of debugging on follow-mode block ingestion, not on P2P peering:

1. Check CL status:
   curl -H "Accept: application/vnd.arc.v1+json" http://127.0.0.1:31000/status

2. Check EL height:
   curl -s -X POST http://127.0.0.1:8545 -H "Content-Type: application/json" -d "{\"jsonrpc\":\"2.0\",\"method\":\"eth_blockNumber\",\"params\":[],\"id\":1}"

3. Capture a longer arc-node-consensus log window after Subscribed to newHeads, to confirm whether blocks are being received/applied or whether the websocket stream drops again immediately.

For troubleshooting, I would also temporarily remove the Blockdaemon follow endpoint so the logs only reflect dRPC + QuickNode behavior.

If both the CL status height and EL block number stay flat after subscription, this looks more like a follow-endpoint ingestion problem (or an EL/CL handoff issue) than a snapshot mismatch or missing P2P peers.

Can a maintainer confirm whether zero connected_peers is expected in --follow mode? The current docs suggest that follow mode should not depend on normal P2P consensus peering for sync.

[Bela1401]

If helpful, we already have a working node setup and would be happy to write a short step-by-step guide for bootstrapping and running an Arc testnet node. If the team is open to it, we can package it as a small docs PR.

[cl-co-dr]

Now i can see that it syncs after restart about 30 blocks and then stops syncing . this is the behaviour on each restart

[cl-co-dr]

arc_con.log

[Bela1401]

I reviewed the new log more closely and the failure now looks more specific than a general sync or peer-connectivity issue.

This does not look like the same root cause as #20, and it also does not look like a generic “no peers” problem:

• the node is clearly running in RPC sync / --follow mode
• it connects and subscribes to newHeads on both QuickNode and dRPC
• it successfully finalizes and commits blocks 37447214 through 37447233
• EL handoff also appears healthy over that span, since forkchoice_updated succeeds through height 37447233

The stall seems to happen at the next contiguous fetch window.

From the attached log, these ranges are fetched successfully:

• 37447224..=37447233
• 37447244..=37447253
• 37447254..=37447263

But 37447234..=37447243 repeatedly fails with Fetch failed, after which the follower keeps retrying and the local tip stays pinned at 37447233.

That same failing range is then retried against both public follow endpoints, and both eventually return invalid responses for it.

So my current read is:

• not a snapshot bootstrap problem
• not a generic websocket subscription problem
• not normal P2P peering
• more likely either:
  1. a bad/malformed block or certificate somewhere in 37447234..=37447243, or
  2. a v0.6.0 follow-mode batching/parsing bug triggered by something in that range

Can a maintainer check whether there is anything unusual about heights 37447234..=37447243, or whether the public follow endpoints are serving a batch in that range that current follower logic rejects?

[cl-co-dr]

It's syncing now in --follow mode but is follow mode enough to run a rpc node serving requests ?

[Bela1401]

Yes, based on the current docs, --follow mode is enough for running a read-only RPC node.

The JSON-RPC endpoint is served by the execution layer, while the consensus layer in --follow mode keeps it synced by fetching finalized blocks from trusted endpoints. The official docs’ quick-start uses exactly that setup, and the consensus README explicitly says follow mode is useful for read-only nodes.

So if the goal is to serve normal RPC requests, --follow should be sufficient.

The main limitation is that this is not the same as participating in P2P consensus or running as a validator. Also, in the example config, --rpc.forwarder is enabled on the EL, so some methods may still be proxied upstream if they are not served locally.

[melekes]

Can a maintainer confirm whether zero connected_peers is expected in --follow mode? The current docs suggest that follow mode should not depend on normal P2P consensus peering for sync.

yes, that's expected ✅

For troubleshooting, I would also temporarily remove the Blockdaemon follow endpoint so the logs only reflect dRPC + QuickNode behavior.

The WebSocket URL for Blockdaemon was incorrect. Please use the following:

  --follow.endpoint https://rpc.blockdaemon.testnet.arc.network,wss=rpc.blockdaemon.testnet.arc.network/websocket \