[Draft] feat(vardiff): add in-process simulation framework + baseline regression tests

[gimballock]

Adds a deterministic in-process simulation framework that characterizes
any Vardiff implementation across the operational rate range, and
commits the current algorithm's measurements as a baseline for automated
regression testing.

The "vardiff fires too often on noise" / "vardiff doesn't react fast
enough" conversations have circled the same issues (#396 and adjacent)
without a way to settle questions empirically. This PR adds the missing
infrastructure: any future proposal can now produce a quantitative
delta against a fixed reference.

The finding that motivates this

Before the framework, "is the algorithm too noisy or too sluggish?" was
a matter of opinion. With it, the question is a table:

Reaction sensitivity to a -50% step change (probability of firing
within 5 minutes after the change):

share/min	sensitivity
6	0.70
12	0.55
30	0.33
60	0.16
120	0.09

Higher share rates produce less responsive algorithms — counter to
expectations. The mechanism is mechanical (post-convergence delta_time
grows indefinitely, diluting the post-step signal in the cumulative
window) and surfaced clearly in the data despite taking months of
deployment observation to half-articulate. Full numbers and analysis
in vardiff_baseline.md.

This isn't a fix. It's the measurement that lets the fix be evaluated.

What's in the PR

3 commits, ~2000 LOC plus baseline data:

1. feat(vardiff): inject Clock trait + add_shares trait method
   Minimum API additions to channels_sv2 for testability and
   simulation performance. Production behavior unchanged — existing
   constructors default to SystemClock, the new trait method has a
   default implementation that keeps existing impls compiling.

2. feat(vardiff_sim): in-process simulation framework
   New crate at sv2/channels-sv2/sim/. Per-tick Poisson share
   sampling, five behavioral metrics with percentile distributions,
   50-cell parameterized sweep (5 share rates × 10 scenarios), CLI
   binary for baseline generation, regression test asserting against
   a committed baseline.

3. data(vardiff_sim): design doc + baseline characterization
   The design proposal documenting metric definitions and tolerance
   policy, plus the measured baseline as both TOML (consumed by the
   regression test) and Markdown (for human review).

What the framework measures

Five behavioral attributes, each as a distribution across 1000
independent trials per cell:

Metric	Better is	What it tells you
Convergence time	Smaller	How fast the algorithm settles after cold start
Settled accuracy	Smaller	How close to truth the algorithm lands
Steady-state jitter	Smaller	How often it fires on noise post-settle
Reaction time	Smaller	How fast it responds to genuine load changes
Reaction sensitivity	≈ 1 for real Δ, ≈ 0 for noise	Whether it distinguishes signal from noise

Per-metric tolerances are asserted automatically against the checked-in
baseline. Failed assertions identify the cell and metric with specific
baseline-vs-current numbers. Mid-range Δ values (10-25%) are reported
but not asserted — that's where legitimate algorithmic tradeoffs live
and a reviewer should judge by looking at the full delta.

How to run

From sv2/channels-sv2/sim/:

# Fast unit tests (~1 second)
cargo test

# Generate a fresh baseline (~5-15 seconds)
cargo run --release --bin generate-baseline

# Run the slow regression test (~5-15 seconds; #[ignore]-d by default)
cargo test --release --lib -- --ignored

What this enables

For any future vardiff proposal:

1. Implement the new algorithm as a Vardiff impl
2. cargo run --release --bin generate-baseline to produce comparable
   measurements
3. Diff against the committed baseline
4. Make the case with numbers

No more "I think this is better." Instead "this changes p50 jitter from
X to Y at 12 spm, at the cost of p90 reaction time going from A to B."

Where to look in this PR

• Design proposal (architecture, metric definitions, tolerance
  rationale): sv2/channels-sv2/sim/VARDIFF_SIMULATION_FRAMEWORK.md
• Crate README (usage, output interpretation, baseline-update
  workflow): sv2/channels-sv2/sim/README.md
• The current algorithm's measured baseline:
  sv2/channels-sv2/sim/vardiff_baseline.md

What this PR is NOT

• Not an algorithm change. VardiffState behavior is unchanged.
  The only public-API additions are Vardiff::add_shares (with a
  default impl) and the Clock trait. Production code defaults to
  SystemClock and behaves identically to before.
• Not a recommendation about share rate defaults. The baseline
  data suggests 12-30 spm is the operational sweet spot, but this
  PR doesn't touch any defaults.
• Not a CI workflow. The regression test works locally but needs
  a GitHub Action to be a true CI gate. Follow-up.

Open follow-ups

• Wire cargo test --release --lib -- --ignored into CI on PRs
  touching vardiff/* or the sim crate.
• Bump channels_sv2 5.0.0 → 5.1.0 once the workspace lockfile
  situation allows (the trait-method addition is technically a
  minor-version semver change). TODO comment in Cargo.toml tracks
  this.
• Investigate the reactivity-degrades-with-rate finding. The framework
  surfaces the problem; fixing it is a separate proposal that this
  framework will be the right tool to evaluate.

Test plan

• cargo test -p channels_sv2 --lib vardiff — 17 tests, all pass
• cargo test from sv2/channels-sv2/sim/ — 53 fast unit tests
• cargo test --release --lib -- --ignored from sim/ — slow
  regression test passes against committed baseline
• cargo run --release --bin generate-baseline — reproduces the
  committed vardiff_baseline.toml byte-for-byte at the same seed

[gimballock]

The code is cheap and only meant to demonstrate the feasibility, but the concept ack revolves around these points imo:

• We can play dice with share-received events to simulate running the vardiff algorithms over arbitrary ranges of time. But we need to mock SystemTime::now() and add a way to bulk add new shares.
• With fake time simulations we can do large scale vardiff trials of whatever metrics we want and contrast against correlated attributes like target shares-per-minute.
  • I was interested in convergence time, stable-state jitter, and convergence accuracy
  • But responsiveness to external change is also a key capability, (how fast to adjust to a 50% spike/dip in hashrate)
• With this compilation of reproducible test results compiled into a profile we can use integration tests to lock in established performance thresholds and ratchet up the expectations if we find better algorithms.

[gimballock]

The first row here shows results of the convergence time test for the the default case (6 spm).
The convergence times are between 10 and 25 minutes, with total failures to converge (w/in quiet_window_secs of simulated time) occurring 17% of the time!

the next few rows describe the results for faster share rates. the most extreme times (25m reduces to 15m) and the total failure cases generally disappear around 30 spm.

[gimballock]

These two metrics (proximity to true hashrate and post-converged adjustments) show a similar trend,
Lots of undesired behavior in the extreme cases (top 10%, top 1%) of 6 shares/min case that is alleviated at higher share rates.

[gimballock]

These tables show how long it takes for vardiff to respond to an unexpected change in hashrate. Where the changes are to either increase or decrease by proportional amounts anywhere from 5% to 50%.

The first table specifically looks at a 50% draw down showing that a full 30% of the time vardiff fails to adjust after 5 min. The next few rows show that the situation worsens at higher share rates, at 120 spm 91% of the trials failed to adjust after 5m.

The second table shows that this effect is basically the same for hashrate changes in the opposite direction and also that changes of lesser magnitude respond much more quickly.

[gimballock]

Convergence rate: Must be no more than 1% slower than the baseline convergence time
Convergence p90: The slowest 10% convergence times must be within 10% of the baseline's convergence time
Settled accuracy: must be within 15% of baseline's accuracy for the slowest 50% / 10%
Jitter p50/p95: must be within 2% and 25% of baseline
...etc.

You see the pattern, there are lots of magic thresholds in this portion of the code that are arbitrarily chosen at this point and fair game for analysis.

[adammwest]

after some optimization I got
https://github.com/adammwest/stratum/blob/feat/vardiff_kalman/sv2/channels-sv2/sim/vardiff_best.md
with the Bayesian model

Method	Result
Bayesian model	Good, Best result
Kalman filters	Good
Jurik moving averages	Good but artifacts on hashrate changes
Thompson sampling	Bad

[gimballock]

after some optimization I got https://github.com/adammwest/stratum/blob/feat/vardiff_kalman/sv2/channels-sv2/sim/vardiff_best.md with the Bayesian model
Method Result
Bayesian model Good, Best result
Kalman filters Good
Jurik moving averages Good but artifacts on hashrate changes
Thompson sampling Bad

I'm so excited to see people other people nerding out on vardiff with me! Thank you!

A couple things I noticed in your results, the 2m convergence time is impressive but your response to a 50% hashrate drop only succeeds in readjusting 4.4% of the time. I'm not sure how best to balance those two metrics but probably not one at the expense of the other.

[adammwest]

Some learning's I had @gimballock

• This task is hard, I think the current implementation is optimized to a degree.
• the most critical thing is the fitness, currently there are many metrics, how you combine all of them into a final value is what determines the goodness for any algorithm, as its a summary there are ways to game it.
• Use every value in the toml file, if you dont those values will naturally will degrade.
• There are many ways to combine many numbers which lead to slightly better and slightly worse performance.
  one thing I did which helped was to separate fitness into 2 categories improvement and regression, even better separating these per group e.g stable,coldstart ,... then you can decompose the value.
• Normalize each metric/group otherwise the more numerous or larger numbers will be the focus.
• There are many cases where you can get a good score, but the fitness prefers optimizing 1 variable or a set of variables at the expense of others.
• For grid parameter sweeps, they usually discretize the domain so you are bounded in improvement only by dimension range and amount of queries. so you need to constantly increase queries or shrink ranges. usually you are limited due to time. For this reason I prefer random restart hill climbing I find is generally pretty good when you don't make assumptions about the data.
• If you have too many parameters to optimize you can over fit, and end up just gaming the test

[gimballock]

Thanks for these insights @adammwest — especially the point about fitness decomposition and normalization. A lot of what you're describing matches the evolution I've gone through on this PR, so let me give a timeline of
how the approach has matured:

Phase 1: Basic metrics + simulation harness

Initially I focused on three metrics I thought were important: convergence time, jitter, and accuracy. These were evaluated via a time-compressed simulation that replays a synthetic share stream through the vardiff
algorithm. This gave us reproducible, large-scale trials (50 cells × 1000 trials) against correlated attributes like target shares-per-minute.

Phase 2: Decomposed pipeline model

I wanted to make algorithm search more systematic, so I decomposed "a vardiff algorithm" into four independent, replaceable components: estimator, statistic, boundary, and decision rule. The idea was to mix-and-match
implementations at each slot for the best composite.

This model worked well for the classic algorithm, the parametric variant, and the EWMA approach. But when I tried to embed a Bayesian model, it broke down — the components aren't truly independent. There's a sequential
data flow: the estimator needs to communicate its belief to the boundary ("should we respond?") and to the update rule. Additionally, since vardiff triggers on a timer rather than on share arrival, the decision rule needs
to call back to the estimator to update state when adjustments occur. I also dropped the "statistic" component as it wasn't pulling its weight.

The resulting three-stage pipeline (Estimator → Boundary → UpdateRule) is what's in this PR. It successfully hosts the classic algorithm, EWMA, AdaCUSUM, and could host a Bayesian approach.

Phase 3: Aggregate fitness metric

To your point about "how you combine all metrics into a final value" — we now have a configurable aggregate metric that allows weighting across the underlying measurements. This addresses exactly the gaming concern you
raised: rather than optimizing one metric at the expense of others, we can define a weighted composite that represents our desired tradeoff. The regression baseline locks in the full vector of metrics so we catch
regressions in any dimension, not just the aggregate.

Your suggestion to separate fitness into improvement vs. regression categories per scenario group (stable, coldstart, reaction) is a good one. Currently the regression test does compare per-cell, so a coldstart regression
can't hide behind a stable-state improvement, but making this more explicit in the scoring would help.

Phase 4: Realistic operating conditions

After discussions with hardware engineers, I retuned the test scenarios to realistic share rates (2–30 spm instead of the earlier 6–120 range). The engineers confirmed that responding to partial hardware failures and
network slowdowns on an established channel is valuable functionality — even though in practice many hashrate changes currently cause miner reconnections (which resets vardiff anyway). We've been doing live testing with
physical miners on testnet4 and confirmed this pattern: when vardiff ramps difficulty too aggressively, it can interact with firmware timeout behaviors in ways that force reconnections, making reactivity testing harder
than expected.

Current direction

I've backed off from prioritizing convergence speed after seeing overcorrection in practice. The current focus is on:

• Stability under steady-state (minimal oscillation once converged)
• Reasonable reactivity (detect genuine changes within 2–3 retarget windows, not 1)
• Asymmetric cost awareness — difficulty increases are more disruptive than decreases. An overshoot upward causes difficulty-too-low share rejections (wasted miner work), while an undershoot downward just means slightly
  more shares than optimal (cheap). The AsymmetricCusumBoundary encodes this: it requires stronger evidence before raising difficulty than lowering it. We can now actually measure the impact via the share-rejection metrics
  (shares_rejected_total{reason="difficulty-too-low"}) that were recently added to the pool's monitoring (sv2-apps PR Docs: Channel Factory #491).

On your point about normalization: agreed, and the per-metric tolerance budgets in the regression test (absolute slack + optional multiplicative slack) are our current mechanism for this. Open to suggestions on better
normalization approaches

[gimballock]

I've been working on a simple proxy to validate the vardiff responsiveness findings from the simulations. My first test aimed to confirm the assessment that the existing vardiff algorithm is slow to respond to hashrate changes after it has already converged.

Test Setup:

I configured an S21 miner → tproxy (vardiff disabled) → shape-proxy (controllable share rate) → SRI pool. The shape-proxy maintains a smoothed share rate based on current pool difficulty and can selectively drop shares to simulate hashrate changes on command.

Methodology:

I ran two parallel instances of the unmodified SRI pool server (main branch) and triggered a 50% share rate drop via API command. I then measured how long it took for the pool's hashrate estimate and share acceptance rate to adjust to the new rate.

Results:

The pool required approximately 40 minutes to complete the initial hashrate adjustment, while the simulation predicted 70% of miners failed to adjust at all and of those that did it took 5m. So it may look like a 20× discrepancy between simulation and observed behavior, its more of a confirmation that it takes a very long time.

I will investigate if there is any discrepancy and re-calibrate the simulation if so. The attached image shows the resulting hashrate and accepted share rate for both trials.

The blue lines are the pool hashrate, you see the initial spike after it first starts up is the convergence spike then is settles and stays flat for not-quite an hour before I activate the 50% hashrate drop, immediately visible in the share rate drop. This stays flat until eventually vardiff fires the adjustment allowing the share rate to return to normal.

[gimballock]

Here you can see the previous 12h of consistent hashrate w/ zero rejected shares as evidence that the 'smoothing' of the hashrate via my proxy is stable enough to test against.

[paratoxicdev]

Hi, this is quite a detailed analysis and great decomposition of relevant parts.

Have you had a look at how ckpool implements this? I think he has quite naturally arrived at a very optimal state, balancing the different metrics. This is the repo, you'll have to grep through to find the vardiff implementation: https://github.com/ckolivas/ckpool

I've re-implemented his approach in Rust as well and made it a bit more configurable here: https://github.com/parasitepool/para/blob/master/src/vardiff.rs

Would be interesting to see how that algorithm performs in your benchmarks.

[gimballock]

Hi, this is quite a detailed analysis and great decomposition of relevant parts.

Have you had a look at how ckpool implements this? I think he has quite naturally arrived at a very optimal state, balancing the different metrics. This is the repo, you'll have to grep through to find the vardiff implementation: https://github.com/ckolivas/ckpool

I've re-implemented his approach in Rust as well and made it a bit more configurable here: https://github.com/parasitepool/para/blob/master/src/vardiff.rs

Would be interesting to see how that algorithm performs in your benchmarks.

Thanks for the info @paratoxicdev , I will add this to my investigation. I know that is a sv1 native pool but i will see what bits of his research crossover to sv2 context and see if it's competitive!

[gimballock]

Here is a breakdown of the calibration comparisons I made between the real hashrate tests and the simulation results confirming the predictions with the understanding that; with this algorithm responsiveness scales with the age of the connection. Suggesting that this detail be included in the simulation so metrics are more directly comparable:

⏺ Vardiff Calibration Summary: Simulation vs Real Miner Testing

  Algorithm under test: Classic Parametric vardiff (VardiffState from stratum-core, branch vardiff/parametric-thresholds)

  Hardware: Antminer S21 (~200 TH/s), testnet4
  Pool config: shares_per_minute = 6 and shares_per_minute = 20
  Tool: shape-proxy with Step{1.0→0.5, at_secs:300} and Track{1.0} reset

  ---
  Structural findings confirmed
  
  ┌────────────────────────────┬───────────────────────────────────────────────────────┬────────────────────────────────────────────────────────┬───────┐
  │          Property          │                    Sim prediction                     │                    Real observation                    │ Match │
  ├────────────────────────────┼───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────┼───────┤
  │ Steady-state jitter        │ 0.000 fires/min                                       │ Zero fires during 30+ min stable operation (both       │ Yes   │
  │                            │                                                       │ rates)                                                 │       │
  ├────────────────────────────┼───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────┼───────┤
  │ Per-fire step magnitude    │ Deterministic: realized_spm / target_spm ratio        │ -16.7% consistently (5 consecutive fires)              │ Yes   │
  ├────────────────────────────┼───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────┼───────┤
  │ Fire cadence after counter │ Exactly 300s (15% threshold at delta_time≥300)        │ 22:39→22:44→22:49→22:54→22:59 (5-min cadence)          │ Yes   │
  │  reset                     │                                                       │                                                        │       │
  ├────────────────────────────┼───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────┼───────┤
  │ Overshoot after staircase  │ p99 = 69% at 6 spm                                    │ ~60% overshoot → share flood → oscillation             │ Yes   │
  │ descent                    │                                                       │                                                        │       │
  ├────────────────────────────┼───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────┼───────┤
  │ Sensitivity depends on     │ Implicit in algorithm (accumulating counter never     │ 5-min counter: 4.4 min reaction; 51-min counter: 51.8  │ Yes   │
  │ counter age                │ resets except on fire)                                │ min reaction                                           │       │
  ├────────────────────────────┼───────────────────────────────────────────────────────┼────────────────────────────────────────────────────────┼───────┤
  │ Algorithm symmetric for    │ Sim shows similar reaction rates both directions      │ Confirmed: similar timescales for step-down and        │ Yes   │
  │ ±50%                       │                                                       │ step-up                                                │       │
  └────────────────────────────┴───────────────────────────────────────────────────────┴────────────────────────────────────────────────────────┴───────┘

  ---
  Reaction time: -50% step (hashrate halved)
  
  ┌────────────────────────────────────┬──────────────────────────────────────────────────────────┬──────────────────────────────────┐
  │             Condition              │                      Sim prediction                      │           Real result            │
  ├────────────────────────────────────┼──────────────────────────────────────────────────────────┼──────────────────────────────────┤
  │ 6 spm, fresh counter (~5 min)      │ 3.7% fire within 5 min; those that fire: p50=5m          │ Slot 3: 4.4 min, Slot 4: 8.0 min │
  ├────────────────────────────────────┼──────────────────────────────────────────────────────────┼──────────────────────────────────┤
  │ 6 spm, settled counter (~51 min)   │ ~96% don't fire within 5 min (tail extends indefinitely) │ Slot 3: 51.8 min                 │
  ├────────────────────────────────────┼──────────────────────────────────────────────────────────┼──────────────────────────────────┤
  │ 20 spm, moderate counter (~27 min) │ 14.2% fire within 5 min                                  │ Slot 3: 6.9 min, Slot 4: 8.4 min │
  └────────────────────────────────────┴──────────────────────────────────────────────────────────┴──────────────────────────────────┘

  ---
  Reaction time: +50% step (return to full rate)
  
  ┌───────────────────────────┬──────────────────────────────────┬──────────────────────────────────┐
  │         Condition         │          Sim prediction          │           Real result            │
  ├───────────────────────────┼──────────────────────────────────┼──────────────────────────────────┤
  │ 6 spm, 51-min counter     │ Deep in tail (>96% non-reactive) │ Slot 3: 51.8 min                 │
  ├───────────────────────────┼──────────────────────────────────┼──────────────────────────────────┤
  │ 6 spm, 68-min counter     │ Deep in tail                     │ Slot 4: 15.4 min                 │
  ├───────────────────────────┼──────────────────────────────────┼──────────────────────────────────┤
  │ 20 spm, 20-21 min counter │ Higher spm improves detection    │ Slot 3: 5.9 min, Slot 4: 6.3 min │
  └───────────────────────────┴──────────────────────────────────┴──────────────────────────────────┘

  ---
  Key insight: counter age is the dominant variable
  
  The sim's test design (step at t=15min, 5 min after cold-start convergence) always tests with a fresh counter. This produces the "3.7% react within 5 min"
  figure. In reality, a pool that hasn't fired in hours has a massive accumulated counter that dilutes any step signal — explaining 40-minute to 2.5-hour
  response times observed in earlier tests.

  At 20 spm vs 6 spm with the same counter age (~20 min), reaction time improves ~8x (51.8 min → 5.9 min). This is because more shares per minute means the
  new rate accumulates statistical weight faster against the counter history.

  ---
  Simulation validity assessment
  
  The simulation is trustworthy for relative algorithm comparisons. It correctly models:
  - The threshold table mechanics (fire/no-fire decisions)
  - The deterministic staircase behavior post-fire
  - The zero-jitter steady state
  - The share-rate dependence on detection speed
  - The fundamental sensitivity-decay-over-time flaw

  Gap to address: The sim should add a "counter age" axis (test steps at t=30m, t=60m, t=120m) to characterize the tail distribution, which is where
  real-world pools spend most of their time. The current 5-minute observation window and 15-minute step timing understate the algorithm's poor real-world
  responsiveness.```  

[gimballock]

Ok here is evidence that the top algorithm (deployed side-by-side with the current vardiff) is immune to the age-dependence effect. This reproduces results predicted in the simulation but now seen in real life.

I started a mining channel against both pools and let them mine overnight.
In the morning I dropped the hashrate in half for both pools at roughly the same time and waited till both pool's vardiffs finished adjusting to the new hashrate changes. The annotated image below shows the results from the grafana dashboard.

The current vardiff algorithm not only took several hours to respond, the response it eventually made was in the wrong direction! While the new algorithm responded in a few minutes and settled on the correct value.

[paratoxicdev]

Thanks for analysis @gimballock!

Why not consider share-based vardiff, if you've used it before?

Since you're also Rust-based I would have a look at the vardiff.rs + decay.rs files with Claude. The type system has allowed much better encapsulation and reasoning about behaviour and I think that'll be easier to compare than the C code from ckpool.

I do have to disagree with some of the things your Claude said:

Where timer-based genuinely wins: Scalability. In SV2 with multiplexed channels, evaluating vardiff on every share means the hot path (share validation → channel dispatch → vardiff state update)
gets heavier per share. Timer-based decouples evaluation from the ingestion path — shares go into a counter, evaluation happens on a separate cadence.

Each channel has their own Vardiff so there's no shared data structure with contention. The pool is overall O(shares), where a couple of floating point operations for share based vardiff are completely dwarfed by the cost to deserialize a share from the wire and then do a double sha256 hash for nonce validation (which is done no matter what Vardiff you use). I'm not that caught up on SV2, so please correct me if that's not what it does.

The real answer: Timer-based with a well-tuned boundary (PoissonCI or CUSUM) is strictly better for pool architecture. The "responsiveness" argument for share-based evaluation is illusory — it's
the boundary's evidence threshold that gates reaction time, not how often you ask the question. Asking more often with less data per ask doesn't help.

So the ckpool algo not only specifies an estimator (exponentially weighted moving average, see decay.rs) but also a boundary (when to adjust difficulty). The two main parts which make up the boundary are the HYSTERESIS bounds and MIN_WINDOW_RATIO.

/// Minimum window ratio before considering adjustment.
/// Fraction of expected time (or shares) per window.
/// Derived from ckpool: 240s / 300s window = 0.8
const MIN_WINDOW_RATIO: f64 = 0.8;

/// Only decrease difficulty when rate drops below this fraction of target.
/// Copied from ckpool.
const HYSTERESIS_LOW: f64 = 0.5;

/// Only increase difficulty when rate exceeds this fraction of target.
/// Copied from ckpool.
const HYSTERESIS_HIGH: f64 = 1.33;

#[derive(Debug, Clone)]
pub(crate) struct Vardiff {
    period: Duration,
    window: Duration,
    min_shares_for_adjustment: u32,
    min_time_for_adjustment: Duration,
    dsps: DecayingAverage,
    current_diff: Difficulty,
    old_diff: Difficulty,
    first_share: Option<Instant>,
    last_diff_change: Instant,
    shares_since_change: u32,
    min_diff: Option<Difficulty>,
    max_diff: Option<Difficulty>,
    diff_change_job_id: Option<JobId>,
}

So not only is the responsiveness tuneable, you can see its responsiveness working on real mining machines in the wild: https://stats.ckpool.org/. From experience I can say that it responds beautifully to any scale of hashrate, be it a cpu miner, a bitaxe, or 1 EH/s rental hashrate.

I really like the theoretical work you're doing and it has deepened my understanding ( (I now know words like estimator and boundary) of what I originally just copied from ckpool. I feel like this algo should be the benchmark that any new vardiff is compared with, since its proven to work with real hashrate.

That's just my two cents, like the work you're doing!

[gimballock]

@paratoxicdev You're right on both points — I should have posted my actual findings rather than a raw Claude transcript.

On scalability: You're correct. Each channel owns its own Vardiff — no shared state, no contention. The FP cost of decay_time() is trivial next to deserialize + SHA256d. Timer-based in SRI is an architectural choice (the pool already runs a 60s tick loop), not a performance win.

On "strictly better": Also wrong. The reality is more nuanced than either of us stated — I'll explain below.

What I found after reading your vardiff.rs + decay.rs:

I ported ckpool's algorithm and benchmarked it across SPM 4–30. Full writeup in commit 6aa27a9, updated in b77eff8.

On your code specifically:

1. Your single-window DecayingAverage is simpler than ckpool's dual-window. My sweep confirmed — once τ_long is shortened to match our tick cadence, the dual-window switching adds nothing. You already knew this.

2. Your time_bias uses time_since_first (miner lifetime), not time since last retarget. This is critical — for any miner running >5 minutes, bias ≈ 1.0 and it's effectively inert. When I naively reset bias on each retarget (as our pipeline's on_fire() does), I got 1 / (1 - e^(-60/300)) ≈ 5.5× amplification on the first tick after every difficulty change — catastrophic overshoot (177–275% settled accuracy). The fix was to simulate per-share decay_time() calls within each tick, letting the EMA warm up organically.

3. Your value_at() (read-without-mutate) is a clean pattern — prevents observation from altering state.

Why we didn't use ckpool's components in production:

I tested all three stages of the pipeline (estimator, boundary, update) independently:

Estimator: After correct porting via per-share simulation, CkpoolEstimator initially appeared equivalent to EwmaEstimator(120s) — they matched within ~5% under a conservative boundary (PoissonCI). But when I paired both with our production-tuned boundary (which uses CUSUM for high-SPM miners), the ckpool estimator underperformed significantly:

Composition	SPM 4	SPM 8	SPM 12	SPM 20	SPM 30
EwmaEstimator(120s) + tuned boundary	0.689	0.768	0.787	0.882	0.869
CkpoolEstimator(60,300) + same boundary	0.698	0.638	0.696	0.777	0.858
CkpoolEstimator(60,120) + same boundary	0.598	0.688	0.764	0.805	0.858

The per-share decay_time() simulation running N decay steps per tick introduces more per-tick noise than a single batch EWMA update. Under the conservative PoissonCI boundary this noise is masked (the boundary requires large deviations to fire). Under CUSUM (which fires on smaller accumulated deviations), the extra noise triggers more false fires → higher jitter → lower fitness. So the evaluation cadence isn't purely scheduling — the numerical stability of the estimate depends on how the information is processed.

Boundary: I swept hysteresis from your native [0.5, 1.33] through [0.9, 1.1] with varying data gates:

Boundary	SPM 4	SPM 10	SPM 20	SPM 30	Jitter (SPM 10)
AdaptivePoissonCusum (production)	0.689	0.774	0.882	0.869	0.060/min
Hyst [0.7, 1.3]	0.621	0.695	0.666	0.608	0.043/min
Hyst [0.8, 1.2]	0.572	0.675	0.723	0.726	0.152/min
Hyst [0.85, 1.15]	0.578	0.634	0.706	0.730	0.254/min
Hyst [0.5, 1.33] (native)	0.630	0.542	0.508	0.504	0.005/min

Narrower bands get excellent reaction rates (96–100%, better than anything else), but at 5–10× jitter cost. The fundamental issue: hysteresis fires whenever the rate ratio crosses the band, with no evidence accumulation. Statistical boundaries (PoissonCI, CUSUM) require cumulative evidence before firing, so they distinguish real changes from noise.

Update: ckpool's full retarget (η=1.0) overshoots in the tick framework. Our AcceleratingPartialRetarget (η ramps 0.2→0.4) achieves the same convergence speed without overshoot.

What we ended up with:

The investigation led to a new production composition that adapts its boundary strategy based on the miner's share rate — conservative (PoissonCI) for low-SPM miners where data is sparse, aggressive (CUSUM) for high-SPM miners where evidence is abundant. This is analogous to what ckpool does with its dual-window adaptive switching, just at the boundary layer instead of the estimator layer. Your algorithm's idea of "be conservative when data is sparse, aggressive when data is abundant" is exactly right — we just implement it differently.

SPM	Old VardiffState	New Production	Improvement
4	0.636	0.706	+11%
8	0.737	0.771	+5%
12	0.787	0.793	+1%
20	0.882	0.894	+1%
30	0.869	0.874	+1%

CkpoolEstimator is preserved in the simulation grid as a benchmark contender — it runs alongside every other algorithm in our characterization suite and any future changes are measured against it.

[gimballock]

If you have any idea or suggestions to improve the comparison, let me know.