Skip to content

feat(scorer): add reason sub-label to profit_scored_total#149

Merged
Pablosinyores merged 1 commit into
developfrom
feat/scorer-decision-reason-label
May 20, 2026
Merged

feat(scorer): add reason sub-label to profit_scored_total#149
Pablosinyores merged 1 commit into
developfrom
feat/scorer-decision-reason-label

Conversation

@Pablosinyores

Copy link
Copy Markdown
Owner

Summary

`aether_mempool_profit_scored_total` was labelled only by `decision`, so
the dashboard could see "10 reverted" but not whether they came from
the V2 U256 walker (PR #136), the f64 absurdity floor (PR #136), the
V3 revm verifier (PR #144), or organic revm reverts. Add a `reason`
sub-label distinguishing those code paths so PR #146's decision-pie

  • scored-rate panels can split-by-reason.

Stacks on PR #148 (`feat/dedupe-replay-scorer-helpers`).

Five reason values

Wire labels pinned by `reason_constants_are_stable_wire_labels`:

Reason When it fires Pairs with
`n/a` non-reverted decisions, `no_path`, or any path with no sub-source worth distinguishing any
`u256_walker` V2-only exact-U256 walker (`verify_cycle_u256`) reached a verdict (PR #136) profitable / unprofitable / reverted
`absurdity_floor` f64 fallback above `MAX_PLAUSIBLE_F64_NET_WEI` (1 ETH) downgraded to reverted (PR #136) reverted
`revm_verdict` V3-touching revm sim (`verify_cycle_revm`) ran to completion with a non-reverting verdict (PR #144) profitable / unprofitable
`revm_revert` V3-touching revm sim explicitly reverted/halted (PR #144) reverted

Wire-format

  • Metric: `aether_mempool_profit_scored_total{decision, reason}` —
    cardinality bounded at ~10 `(decision, reason)` combinations.
  • Persistence: `reason` is Prometheus-only. It does NOT flow into
    the `mempool_profitability` Postgres table because the migration's
    CHECK constraint only covers `decision`. `NewProfitabilityScore.reason`
    is consumed by `PgProfitabilityWriter::insert_score` for the metric
    label and dropped before the SQL bind. No migration needed.

Code changes

`crates/grpc-server/src/profitability_writer.rs` (+62 / −5):

  • New `REASON_*` `pub const` block.
  • `scored_total` `IntCounterVec` bumped from `["decision"]` to
    `["decision", "reason"]`.
  • `NewProfitabilityScore` gains `pub reason: &'static str` with a
    `serde(default = "default_reason")` shim returning `REASON_NA` so
    external callers that haven't been updated still deserialise cleanly.
  • `PgProfitabilityWriter::insert_score` reads `score.reason` before
    the `try_send` and uses it in the metric `with_label_values` call.

`crates/grpc-server/src/bin/aether_profit_scorer.rs` (+45 / −19):

  • `ScoreOutcome` gains `reason: &'static str`.
  • `revm_verdict_to_decision` and `f64_fallback_verdict` return a
    4-tuple `(net, realised, decision, reason)`.
  • The aggregating `let` in `score_one` destructures the 4-tuple and
    the U256-walker arm explicitly attaches `REASON_U256_WALKER`.
  • `no_path_outcome` carries `REASON_NA`.
  • `score_batch` plumbs `score.reason` into the
    `NewProfitabilityScore` constructed for the writer.

`deploy/docker/grafana/dashboards/mempool.json` (+8 / −8):

Verification

```
cargo clippy --workspace --all-targets -- -D warnings ✓ clean
cargo test --workspace --lib --bins ✓ 528 passed, 0 failed
python3 -m json.tool mempool.json ✓ parses
```

New unit test:

  • `reason_constants_are_stable_wire_labels` (pinned wire strings)

Existing verdict-helper tests gain `reason` assertions:

  • `revm_verdict_decision_mapping_reverted` → asserts `REASON_REVM_REVERT`
  • `revm_verdict_decision_mapping_profitable` → `REASON_REVM_VERDICT`
  • `revm_verdict_decision_mapping_unprofitable` → `REASON_REVM_VERDICT`
  • `f64_fallback_verdict_above_floor_reverted` → `REASON_ABSURDITY_FLOOR`
  • `f64_fallback_verdict_below_floor_profitable` → `REASON_NA`
  • `f64_fallback_verdict_negative_unprofitable` → `REASON_NA`
  • `no_path_outcome_carries_negative_net_when_gas_given` → `REASON_NA`
  • `metrics_register_round_trips` emits across multiple (decision, reason) pairs.

Operational notes

After this lands, the live scorer process (PID 2476 at handoff time)
needs to restart to pick up the new metric label set. Grafana caches
metric metadata; the panels will start splitting once the new metric
samples arrive. The pre-restart counter series (cardinality
`{decision}` only) coexists with the new `{decision, reason}` series
without breakage — Prometheus treats them as distinct families.

Out of scope

  • `no_path` sub-reasons. The handful of upstream causes
    (pool absent, eth_call empty, optimiser bailed, tokens missing,
    zero reserves) all collapse to `REASON_NA` today. If one proves
    operationally interesting (e.g. sustained `pool_not_in_registry`
    signals a V3 pool-discovery gap), it can be split out in a follow-up.
  • Counter cardinality bound. With 4 decisions × 5 reasons = 20 max
    combinations, the cardinality is well below any Prometheus practical
    limit. No `relabel_config` needed.

@vercel

vercel Bot commented May 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
aether Ready Ready Preview, Comment May 20, 2026 12:20pm
aether-63xv Ready Ready Preview, Comment May 20, 2026 12:20pm

@Pablosinyores Pablosinyores force-pushed the feat/dedupe-replay-scorer-helpers branch from b704f5b to f2cb65c Compare May 20, 2026 12:16
@Pablosinyores Pablosinyores changed the base branch from feat/dedupe-replay-scorer-helpers to develop May 20, 2026 12:17
aether_mempool_profit_scored_total used to be labelled only by
decision, so the dashboard could see "10 reverted" but not whether
they came from the V2 U256 walker, the f64 absurdity floor, the V3
revm verifier, or organic revm reverts. Add a `reason` sub-label
distinguishing those code paths.

Five wire labels (pinned by unit test against the constants):
- n/a              non-reverted decisions, no_path, or any path with
                   no sub-source worth distinguishing
- u256_walker      V2-only exact-U256 walker reached a verdict
                   (PR #136 path)
- absurdity_floor  f64 fallback above MAX_PLAUSIBLE_F64_NET_WEI (1 ETH)
                   downgraded to reverted (PR #136 path)
- revm_verdict     V3-touching revm sim ran to completion with a
                   non-reverting verdict (PR #144 path)
- revm_revert      V3-touching revm sim explicitly reverted/halted
                   (PR #144 path)

The reason is Prometheus-only and NOT persisted to the
mempool_profitability table — the migration's CHECK constraint only
covers decision, and adding a reason column would force every
existing row to back-fill. `NewProfitabilityScore.reason` skips the
DB insert path; it only flows into the metric label.

revm_verdict_to_decision and f64_fallback_verdict now return a
4-tuple (net, realised, decision, reason). no_path_outcome carries
REASON_NA. The aggregating let in score_one destructures
(net, realised, decision, reason) and threads reason into
ScoreOutcome.

Dashboard panels in deploy/docker/grafana/dashboards/mempool.json
(panel IDs 19 and 20 from PR #146) updated to sum by
(decision, reason) and legend-format {{decision}} / {{reason}}.
Title and description updated to reflect the new dimension.

Stacks on feat/dedupe-replay-scorer-helpers (PR #148).

Verification:
- cargo clippy --workspace --all-targets -- -D warnings : clean
- cargo test --workspace --lib --bins : 528 passed, 0 failed
- new test: reason_constants_are_stable_wire_labels
- existing verdict-helper tests updated to assert reason value
- python3 -m json.tool mempool.json : parses cleanly
@Pablosinyores Pablosinyores force-pushed the feat/scorer-decision-reason-label branch from 7b31a6f to e39c717 Compare May 20, 2026 12:17
@Pablosinyores Pablosinyores merged commit b871f2f into develop May 20, 2026
1 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant