Skip to content

harden tProxy Sv1 session and channel lifecycle - #823

Open
GitGab19 wants to merge 26 commits into
stratum-mining:mainfrom
GitGab19:fix/tproxy-session-channel-lifecycle-2
Open

harden tProxy Sv1 session and channel lifecycle#823
GitGab19 wants to merge 26 commits into
stratum-mining:mainfrom
GitGab19:fix/tproxy-session-channel-lifecycle-2

Conversation

@GitGab19

@GitGab19 GitGab19 commented Aug 27, 2026

Copy link
Copy Markdown
Member

This PR is built on top of #741 and:

  • Tracks SV1 setup progress and enables job notifications after both mining.subscribe and mining.authorize complete.
  • Requires upstream jobs to support version rolling.
  • Disconnects SV1 miners when the upstream closes their channel.
  • Applies extranonce prefix changes without recreating channels.
  • Honors mining.extranonce.subscribe, notifying supported miners and disconnecting unsupported ones.
  • Handles malformed mining notifications without panicking.
  • Starts vardiff only after the downstream SV2 channel is open.
  • Retains bounded, job-specific validation state so late SV1 shares use the target and extranonce assigned to their job without treating difficulty updates as clean jobs.
  • Returns standard SV1 rejection errors for missing jobs, duplicate shares, and low-difficulty shares, with bounded per-downstream duplicate tracking.
  • Logs unexpected upstream-channel validation failures while keeping expected target filtering at debug level.

companion stratum-mining/stratum#2319

Closes #36
Closes #139
Closes #164
Closes #386
Closes #650

Comment thread miner-apps/translator/src/lib/sv2/channel_manager/mining_message_handler.rs Outdated
Comment thread miner-apps/translator/src/lib/sv2/channel_manager/mod.rs Outdated
@GitGab19
GitGab19 force-pushed the fix/tproxy-session-channel-lifecycle-2 branch 2 times, most recently from ecfe3ed to 903cebd Compare September 2, 2026 08:39
@bit-aloo

bit-aloo commented Sep 2, 2026

Copy link
Copy Markdown
Member

Clanker review:

Correctness issues

1. mining.set_extranonce can be silently lost during the subscribe window

Locations: downstream.rs:650–700, sv1_server/mod.rs:794

The downstream task decides whether to forward the notification using session_state.is_subscribed() at processing time. However, that flag is flipped by the server task only after the subscribe response has been built and sent.

The race looks like this:

  1. The server builds the subscribe response with the old extranonce1.

  2. The server sends the response.

  3. The downstream task processes the queued SetExtranonce event.

    • is_subscribed == false
    • It updates extranonce1 silently.
    • It drops the cached notification.
  4. The server calls record_response(Subscribe).

At this point, the miner has the old extranonce1, while every new job context is recorded with the new one. All shares from that miner then fail hash validation until it reconnects.

The window is only microseconds, but it creates a correctness hole in exactly the path this PR is trying to harden.

Suggested fix: Make the decision in the server task at enqueue time in handle_upstream_extranonce_change. This is the same task that runs record_response, so there is no race.

For example:

Sv1ServerEvent::SetExtranonce {
    message,
    notify_miner: bool,
}

When the miner is not yet subscribed, eagerly apply extranonce1 / extranonce2_len under the lock so the upcoming subscribe response carries the new value. The event then only preserves FIFO ordering: drop cached_notify and decrement the pending counter.

When the miner is already subscribed, the subscribe response was sent before the event was enqueued, so FIFO guarantees the correct wire order.


2. Aggregated bootstrap job rewinds the shared keepalive history

Locations: sv1_server/mod.rs:1150–1195, sv1_server/mod.rs:1880

A late joiner's bootstrap job (child channel_id) is activate()d into the shared AGGREGATED_CHANNEL_ID store with clean_jobs = false.

Suppose the active job is:

orig#3  (nTime = T0 + 3i)

The bootstrap orig replaces it as active, while orig#3 moves to past.

The next create_keepalive_job then derives from orig and emits:

orig#4  (nTime = T0 + i)

to every miner.

That's the exact slice miners already exhausted under orig#1. They re-search it, and their own accepted_share_hashes cache rejects any rediscovered shares as duplicates.

This was latent before—the old Vec::last() had the same shape—but the new code explicitly special-cases bootstrap jobs, so this seems like the right place to finish fixing it.

Suggested fix: When:

m.channel_id != AGGREGATED_CHANNEL_ID

and the store already has an active job whose original ID equals notify.job_id, don't mutate the shared store. Instead, send the late joiner the current stored active notify with clean_jobs = true.


3. Late aggregated joiners can hang at mining.subscribe for up to a block

Locations: channel_manager/mod.rs:1052–1070, channel_manager/mod.rs:1252

The deferral itself is correct, and the doc comment is honest about the cost: "next block plus one job."

The problem is that SV1 miners typically time out a pending subscribe after ~30–60 seconds and reconnect.

During a prefix transition, this can produce a loop:

subscribe
  → deferred
  → miner times out
  → reconnects
  → subscribe
  → deferred
  → ...

When the compatible job finally arrives, N child channels may be opened and immediately closed through:

DownstreamNotPresent → CloseChannel

This isn't a leak, but it creates unnecessary churn, and the miner sees a connection failure rather than simply a delayed setup.

Possible no-wait alternative

The required building blocks already exist:

  1. Allocate the child prefix.
  2. Call set_upstream_prefix(old_bytes) on it.
  3. Create the ExtendedChannel.
  4. Replay the aggregated channel's active/future jobs, each with its captured prefix, switching the child's upstream region before each replay.
  5. Call set_upstream_extranonce_prefix(new).

The subscribe response then advertises the old variant, and forward_job_to_sv1_server already emits mining.set_extranonce on the first new-prefix job.

Tradeoff: A miner that doesn't support set_extranonce gets one disconnect later instead of waiting now.

Either approach seems reasonable, but it's worth making the choice explicit. If the deferral stays, I'd at least add a bounded timeout that rejects the request with an error so the miner fails fast.


4. A supporting miner can be disconnected mid-handshake

Location: downstream.rs:686

The disconnect currently fires when:

is_subscribed && !supports_set_extranonce

However, mining.extranonce.subscribe commonly arrives after mining.subscribe, and for some firmware it can even arrive after authorize.

A prefix change landing in that window therefore disconnects a miner that would have supported the change once setup completed.

Suggested fix: While Starting, cache the set_extranonce and re-evaluate it at SetupComplete. At that point, either send it after the setup responses or disconnect if the miner still doesn't support it.

Ergonomics / simplification

5. Sv1ServerEvent should carry typed messages, not JSON

Locations: downstream.rs:40, downstream.rs:579–650

The server currently:

  1. Builds a typed Notify.
  2. Converts it to json_rpc::Message.
  3. Sends it through Sv1ServerEvent.
  4. Every downstream task re-parses it with Notify::try_from(notification.clone()).
  5. The downstream also string-matches notification.method.

This happens once per miner per job.

An event enum like:

Notify(Arc<Notify>)
SetDifficulty(Message)
SetExtranonce(SetExtranonce)
SetupComplete

would move JSON conversion to the socket-write boundary.

It would also remove:

  • InvalidMiningNotifyNotification
  • InvalidSetExtranonceNotification
  • the "cached mining.notify was not a notification" branch
  • two of the new unit tests that only cover unparseable internal messages

Also, Sv1ServerEvent::notification(m) is just a one-line alias for:

Sv1ServerEvent::Notification(m)

so I'd drop it.


6. Duplicated extranonce-size arithmetic in handle_set_extranonce_prefix

Locations: mining_message_handler.rs:529, mining_message_handler.rs:623

Both branches manually compute:

upstream + local_prefix_and_index + rollable <= MAX_EXTRANONCE_LEN

before calling:

allocator.set_upstream_prefix(...)
channel.set_upstream_extranonce_prefix(...)

Those methods already perform the same validation transactionally—I verified this in the pinned channels_sv2.

Roughly 60 lines could therefore collapse into mapping ExceedsMaxLength into the appropriate fallback.

The only thing lost is the numeric:

InvalidExtranonceSize {
    prefix_len,
    rollable_size,
}

payload, which can be reconstructed from the channel at the error site if it's still useful.


7. Collision-ID derivation is hard to read

Location: mining_message_handler.rs:88–115

The current code first checks a collision predicate, then uses another nested if to recover which ID actually collided.

A helper like:

fn colliding_channel_id(&self, m) -> Option<ChannelId>

could return the offending ID directly.

That would read more clearly and avoid having the predicate and ID-recovery logic drift apart.


8. handle_set_target_without_vardiff only disconnects the first failed sender

Location: sv1_server/mod.rs:1629

Finishing the broadcast is the right fix, but if two miners are dead, the second one isn't noticed until the next SetTarget.

Since this already runs in the server task, calling:

handle_downstream_disconnect(...)

inline for each failed sender seems simpler than threading a single ID through Action::Disconnect.


9. Sv1AcceptedShareCache::insert_if_new is O(4096) per share

Location: downstream.rs:160

VecDeque::contains scans up to 4096 32-byte hashes on every submit.

This is probably fine at current rates, but if this ever sits in front of many miners, a HashSet + VecDeque pair would give O(1) membership checks while preserving bounded eviction order.

Nits / housekeeping

  • utils.rs:36–62 — The # Arguments / # Returns documentation for validate_sv1_share now attaches to Sv1ShareValidationOutcome, because the enum was inserted between the comment and the function. Move the enum above the doc block.

  • bitcoin-core-sv2/Cargo.toml / stratum-apps/Cargo.toml — These point at a personal fork branch. They should be reverted once the Stratum PR lands, following the same pattern as the codec branch.

  • bitcoin_core_ipc_jdp_io.rs — This is purely a doc-comment reflow and appears unrelated to this change.

  • #[allow(clippy::result_large_err)] on disconnect_downstream_for_upstream_close — This suggests TproxyError has grown fairly large. Boxing the error kind is out of scope here, but probably worth a follow-up.

Document that a new vardiff target takes effect with the next
mining.notify rather than immediately. This preserves the difficulty
advertised with each job so late shares are not checked against a
newer target.
Repeated subscribe or authorize requests must not complete setup more
than once or flush cached mining notifications repeatedly. Track each
response and transition the downstream only on the first complete
handshake.
A failed send to one disconnected miner previously aborted
the broadcast and starved healthy downstreams of their new
difficulty. Finish the fan-out before returning the disconnect action
for the failed peer.
An SV1 miner can pipeline setup traffic while its SV2 channel is
opening. Cap that per-downstream queue at eight messages and disconnect
on the ninth so an unauthenticated peer cannot grow it without bound.
Keep the existing Bitcoin Core IPC integration test aligned with
rustfmt output. This is a formatting-only change with no runtime
behavior impact.
Make the default trust boundary explicit: payout verification is
disabled unless configured, so tProxy otherwise accepts the upstream
payout policy. Operators can then make an informed choice when enabling
the check.
Channel and group identifiers share routing state, so accepting a
reused or reserved identifier can reinterpret later messages for
the wrong owner. Reject collisions transactionally, including the
aggregated sentinel ID.
Aggregated miners share one job history, but independent keepalive
ticks could repeatedly mutate it and deliver different work to
peers. Gate mutation by wall-clock time and distribute one shared
keepalive job to all eligible downstreams.
Temporary dependency override for
stratum-mining/stratum#2319.  Revert to
`branch = "main"` and run `cargo update -p stratum-core` across all
workspaces after the companion PR merges.
The previous atomic handshake flag could not represent subscribe and
authorize progress or preserve response and notification ordering. Use
an explicit session state and release cached mining notifications
only after both setup responses are queued.
tProxy serves typical SV1 miners that may roll block versions, so
an upstream job that forbids version rolling cannot be translated
safely. Treat such work as incompatible and trigger fallback instead
of advertising unusable jobs.
Once the upstream closes a channel, its associated SV1 miner can no
longer submit usable work. Cancel that downstream connection at the
same lifecycle boundary so it reconnects instead of mining against
dead state.
SetExtranoncePrefix is a legitimate in-place channel update, not
evidence that the upstream failed. Apply the new upstream-owned
prefix while preserving local allocation state and keep the existing
channel alive.
Record which downstreams support mining.set_extranonce so that
SetExtranoncePrefix can deliver the subscription notification only
to those miners and disconnect unsupported ones cleanly.
Prefix transitions only need FIFO ordering between
mining.set_extranonce and the first job that uses it. Replace
duplicated transition bookkeeping with one pending count and suppress
keepalives until the matching job is delivered.
Internal notification conversion can fail when a malformed
mining.notify reaches tProxy. Propagate a structured error instead
of panicking so the application follows its normal error policy.
A connected socket is not yet an active mining channel and must not
contribute to vardiff calculations. Register the miner only after
OpenExtendedMiningChannelSuccess establishes the corresponding SV2
channel.
Late shares must be checked with the target, extranonce, and version
advertised for their own job rather than the newest downstream
state. Retain one active job plus bounded history and clear prior
work only on a true clean-jobs transition.
…rent prefix

A late joiner's extranonce1 is minted with the current upstream prefix,
but the aggregated channel may still hold an active or future job
created under a previous prefix. Replaying such a job would make the
new miner produce shares the upstream rejects, so keep the request
pending until every inherited job matches the current prefix, and
flush pending requests when a new job or prev-hash arrives.
Validate shares against each miner advertised work before forwarding
and report stale, duplicate, and low-difficulty failures with their
standard SV1 codes. Expected filtering at the harder upstream target
remains asynchronous in ChannelManager.
Opening an SV2 channel before decoding the first SV1 request lets
malformed or out-of-order traffic allocate upstream resources. Parse
once through sv1_api, enforce only session ordering in tProxy, and
dispatch the queued typed request after channel success.
Exercise clean-jobs behavior through the downstream FIFO, SV1 response
path, and ChannelManager boundary. The coverage ensures a clean
transition invalidates only its recipient and that later difficulty
changes cannot revive stale work.
Parse the first SV1 request through sv1_api before deciding whether to open an upstream channel.

Reply with code 25 when mining.submit arrives before subscription, without allocating a channel or disconnecting the miner. Once subscribed, unauthorized workers receive code 24 through the normal handler and may recover on the same connection.
Adapt tProxy and JDC to the channels_sv2 constructor's full-extranonce validation.

Route invalid upstream layouts through each application's existing fallback or shutdown policy, while test fixtures continue to assert that their fixed layouts are valid.
The companion channels_sv2 API now returns ExtendedChannelError from
upstream-prefix updates. Store that type in the translator error wrapper
and use its Debug representation in diagnostics.

Keep the existing fallback and shutdown decisions at the call sites.
Validated against the local stratum companion, with no dependency or
lockfile changes needed for this adaptation.
Upstream now makes past-job retention configurable. Propagate that same
configuration to shared SV1 jobs and per-downstream validation contexts,
including the channel default for an unset or zero cap.

Exercise caps above and below the default through upstream job delivery
and the downstream FIFO in both aggregation modes, so SV1 validation does
not silently retain a different history from its corresponding channel.
@GitGab19
GitGab19 force-pushed the fix/tproxy-session-channel-lifecycle-2 branch from 903cebd to 2b463bc Compare September 8, 2026 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants