op-supernode: invalidation recovery can repeatedly reset when unsafe sync resumes before EL convergence

[jelias2]

Summary

Invalidation recovery can enter a repeated reset cycle when unsafe synchronization resumes before the execution layer has converged on the deposits-only replacement branch.

A possible sequence is:

1. The supernode detects an invalid executing message.
2. The affected chain is rewound to the invalid block's parent.
3. A deposits-only replacement is requested for the invalidated height.
4. Virtual nodes return to Running.
5. Unsafe synchronization immediately targets a substantially newer unsafe head.
6. The execution layer cannot yet connect that head to the replacement branch and returns SYNCING to a forkchoice update.
7. The derivation pipeline resets.
8. The same invalidation is processed again, restarting global invalidation recovery.

Because invalidation currently freezes all virtual nodes for transitive-invalidation safety, repeated cycles can also reset unaffected chains. Their local-safe heads may walk back approximately one sequencing window and replay repeatedly, even though those chains were not themselves invalidated.

Expected behavior

After an invalid message is detected:

• the affected chain rewinds to the invalid block's parent;
• a deposits-only replacement becomes canonical at the invalidated height;
• the replacement branch and required ancestry become usable by the execution layer;
• unsafe synchronization resumes;
• safe and unsafe progression stabilize;
• the original invalidation does not repeatedly trigger global recovery; and
• unaffected chains do not repeatedly reset or replay local-safe history.

Observed behavior

The same invalidation target can be processed repeatedly after the replacement workflow begins.

Correlated behavior includes:

• repeated insertion of the same block into the denylist;
• repeated rewind initiation and completion;
• deposits-only replacement requests;
• unsafe synchronization requests spanning far ahead of the replacement height;
• forkchoice updates returning SYNCING;
• derivation resets following the unexpected FCU status;
• repeated global VN freeze/reset cycles; and
• deep local-safe walkbacks and replay on unaffected chains.

The invalidation engine rewind itself only targets the invalid block's parent. The much deeper safe-head movement comes from reset-time local-safe recomputation, not from the invalidation rewind depth.

Suspected race

This appears to involve a missing convergence barrier between replacement recovery and resumed unsafe following.

The pending invalidation transition can complete once virtual nodes report Running, but Running may not establish that:

• the deposits-only replacement is canonical on the target EL;
• replacement ancestry is available;
• an FCU against the replacement branch has succeeded;
• local derivation has resumed from that branch; or
• the EL can connect a far-ahead unsafe tip to the replacement branch.

Unsafe synchronization may therefore race ahead of EL convergence. If the EL returns SYNCING, op-node resets to resynchronize with the engine, potentially making the same invalidation visible again.

A second contributing issue may be duplicate invalidation handling. If an invalid block is already denied and its original hash is no longer canonical, the recovery path may still freeze all VNs before determining that no additional rewind is required.

This is a working hypothesis, not a confirmed root cause. A regression reproduction should determine whether the fix belongs in the supernode transition lifecycle, op-node unsafe-sync gating, duplicate-invalidation handling, EL interaction, or a combination.

Relevant code areas

• op-supernode/supernode/activity/interop/interop.go
  • invalidation transition freezes all VNs;
  • transition completion is based on VN lifecycle state.
• op-supernode/supernode/chain_container/invalidation.go
  • denylist insertion and rewind of the invalidated chain.
• op-node/rollup/engine/payload_process.go
  • deposits-only replacement for denied derived payloads.
• op-node/rollup/engine/engine_controller.go
  • local-safe selection when SuperAuthority safe is ahead.
• op-node/rollup/sync/start.go
  • reset-time FindL2Heads local-safe walkback.

Proposed reproduction

Add a test that:

1. Produces an invalid executing message.
2. Keeps the advertised unsafe tip significantly ahead of the invalidated height.
3. Rewinds the affected EL and requests a deposits-only replacement.
4. Causes the EL to return SYNCING if unsafe FCU arrives before replacement-branch convergence.
5. Observes whether derivation resets and the same invalidation is processed again.
6. Includes an unaffected chain to detect repeated global reset/local-safe replay.

The test should distinguish:

• the one-block invalidation rewind;
• deposits-only replacement construction;
• reset-time local-safe recomputation;
• unsafe payload synchronization; and
• duplicate invalidation processing.

Potential fixes to evaluate

Recovery convergence barrier

Do not release normal unsafe following or complete invalidation recovery until the invalidated chain has demonstrated sufficient convergence, potentially including:

• canonical replacement at the invalidated height;
• replacement ancestry available;
• successful FCU on the replacement branch; and
• derivation/local-safe progression resumed from the replacement branch.

The precise barrier should be derived from the regression test and existing lifecycle guarantees.

Duplicate-invalidation preflight

Before globally freezing VNs, check whether every requested invalidation has already been handled:

• the target hash is already denied; and
• that hash is no longer canonical at its height.

If all targets are already handled, avoid restarting global recovery. Preserve the current all-VN freeze whenever a real rewind is still required, since that behavior protects transitive invalidation safety.

Unsafe-sync gating

Prevent a recovering VN from immediately targeting a far-ahead unsafe head until the replacement branch can support that FCU.

This may overlap with the convergence barrier and should not become a separate mechanism unless the test demonstrates that it is necessary.

Acceptance criteria

• A single invalid message results in one stable rewind/replacement recovery cycle.
• The deposits-only replacement becomes canonical at the invalidated height.
• Unsafe synchronization resumes successfully after replacement convergence.
• The same original invalidation does not repeatedly initiate recovery.
• Unaffected chains do not repeatedly reset or walk local-safe backward.
• Temporary EL SYNCING during recovery does not create an unbounded invalidation/reset loop.
• Existing transitive-invalidation safety is preserved.
• A regression test covers a far-ahead unsafe tip and premature FCU during replacement recovery.