CBOR-1: Switch to cbor format in DB by v0d1ch · Pull Request #2767 · cardano-scaling/hydra

v0d1ch · 2026-07-20T10:18:30Z

Summary

Event payloads in the events table of hydra.db were stored as JSON blobs,
paying the hex-in-JSON envelope tax twice: on disk, and in (de)serialization
time on every persisted state change and every replay at startup. This PR
switches the persisted encoding to CBOR.

Measured on a copy of a real 3-node bench database (6,599 transaction events):
27.4 MB → 10.5 MB (2.6× smaller), migration + full replay in 5.3 s.

What changed

Hand-written ToCBOR/FromCBOR instances for the payload closure of
StateChanged (head state, snapshots, chain events, deposits, environment)
and the StateEvent wrapper, following the existing convention: leading text
tag, fields in declaration order. These codecs define the on-disk format
(documented in docs/docs/dev/architecture/event-sourcing.md).
Hydra.Events.SQLiteBased reads and writes rows with
serialize'/decodeFull'; encoding stays on the write-behind thread
(perf-3: Encode SQLite events on the write-behind thread #2779). Schema version bumped to 2.
Automatic migration: opening a v1 database re-encodes all rows to CBOR in a
single transaction with the version bump, then VACUUMs. A row that fails to
decode aborts startup and rolls back to an intact v1 database. No downgrade
path (breaking, in CHANGELOG).
head-state-viewer decodes CBOR with JSON fallback, so v1 archives still open.

Compatibility

Client-visible output is unchanged — rows are decoded into typed values at the
event-store boundary. Peer-to-peer traffic (etcd) was already CBOR.

Format stability in CI

Roundtrip properties for every codec pair (Hydra.CBORSpec), a new
golden/StateEvent.cbor (one sample per constructor) that must keep decoding
and re-encoding byte-identically, and SQLiteBasedSpec tests for the v1→v2
migration and corrupt-row rollback. If the golden test fails, the change breaks
existing databases and needs a schema migration.

CHANGELOG updated or not needed
Documentation updated or not needed
Haddocks updated or not needed
No new TODOs introduced or explained herafter

github-actions · 2026-07-20T10:26:47Z

Transaction cost differences

No cost or size differences found

github-actions · 2026-07-20T10:28:48Z

End-to-end benchmark differences

Comparing this PR (new) against master (old). Numbers come from cloud VMs, so changes under 5% are shown as ≈ and are likely run-to-run noise rather than a real regression or improvement. 🟢 = improvement, 🔴 = regression; uncolored rows are neutral measures reported for context.

Sustained load (3 nodes, 3x5000 txs)

Metric	master	PR	Δ
End-to-end TPS (tx/s)	481.32	517.01	🟢 +35.69 (+7.4%)
Sustained TPS (tx/s)	897.83	1,109.61	🟢 +211.78 (+23.6%)
Backlog drain time (s)	30.40	28.40	🟢 -2.00 (-6.6%)
Snapshots per second (/s)	0.55	0.55	≈ +0.00 (+0.0%)
Avg txs per snapshot	882.40	937.50	+55.10 (+6.2%)
Avg. Confirmation Time (s)	24.762	24.045	≈ -0.717 (-2.9%)
P50 confirmation (s)	26.335	25.091	≈ -1.245 (-4.7%)
P95 confirmation (s)	30.679	28.441	🟢 -2.238 (-7.3%)
P99 confirmation (s)	30.708	28.573	🟢 -2.135 (-7.0%)
Tx validation time p50 (s)	9.352	9.845	🔴 +0.493 (+5.3%)
Peak node RSS (MB)	407.50	350.50	-57.00 (-14.0%)
Invalid txs	0.00	0.00	≈ +0.00 (n/a%)

Plateau 1000 UTxO (1 node, 4000 txs)

Metric	master	PR	Δ
End-to-end TPS (tx/s)	228.52	243.60	🟢 +15.08 (+6.6%)
Backlog drain time (s)	17.40	16.30	🟢 -1.10 (-6.3%)
Snapshots per second (/s)	0.57	0.49	-0.08 (-14.0%)
Avg txs per snapshot	400.00	500.00	+100.00 (+25.0%)
Avg. Confirmation Time (s)	10.093	10.161	≈ +0.068 (+0.7%)
P50 confirmation (s)	9.872	10.332	≈ +0.460 (+4.7%)
P95 confirmation (s)	17.407	16.334	🟢 -1.074 (-6.2%)
P99 confirmation (s)	17.414	16.340	🟢 -1.074 (-6.2%)
Tx validation time p50 (s)	6.204	6.225	≈ +0.021 (+0.3%)
Peak node RSS (MB)	327.90	341.90	≈ +14.00 (+4.3%)
Invalid txs	0.00	0.00	≈ +0.00 (n/a%)

Round-trip latency (3 nodes, closed-loop, 3x250 txs)

Metric	master	PR	Δ
End-to-end TPS (tx/s)	108.07	104.75	≈ -3.32 (-3.1%)
Sustained TPS (tx/s)	109.08	105.05	≈ -4.03 (-3.7%)
Backlog drain time (s)	0.00	0.00	≈ +0.00 (n/a%)
Snapshots per second (/s)	72.62	71.51	≈ -1.11 (-1.5%)
Avg txs per snapshot	1.50	1.50	≈ +0.00 (+0.0%)
Avg. Confirmation Time (s)	0.028	0.029	≈ +0.001 (+3.3%)
P50 confirmation (s)	0.027	0.028	≈ +0.001 (+4.1%)
P95 confirmation (s)	0.038	0.038	≈ +0.000 (+0.0%)
P99 confirmation (s)	0.044	0.044	≈ +0.000 (+0.0%)
Tx validation time p50 (s)	0.007	0.007	≈ +0.000 (+2.8%)
Peak node RSS (MB)	149.00	146.70	≈ -2.30 (-1.5%)
Invalid txs	0.00	0.00	≈ +0.00 (n/a%)

github-actions · 2026-07-20T10:39:53Z

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2026-08-13 15:49:48.235058991 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νHead	f2dd4ade71e19c2310a86215aa78aea06463aca2d8b818af8dc1b8a4	12805
μHead	4abb8dedbcd6a6f03f4fe227300e2713d73b7680d47baa898b60d27a*	4971
νDeposit	c78e8c9205721eb3ef4410f3db9c6169fa6db497c24641d29c20529c	1615
νCRS	09db7ee6cf7a4b358dd5c8a2f19d2c048336ffc5a01ef35a47ca7072	2736

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

`Init` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5473	9.28	3.05	0.49
2	5572	10.02	3.29	0.50
3	5668	10.13	3.31	0.51
5	5860	11.33	3.70	0.53
10	6340	13.64	4.43	0.57
50	10183	35.15	11.16	0.96
100	14987	62.28	19.66	1.45
114	16326	69.29	21.84	1.59

Cost of Increment Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	2320	21.24	7.55	0.48
2	2452	22.31	8.56	0.50
3	2583	23.10	9.47	0.52
5	2845	25.21	11.48	0.56
10	3498	30.23	16.40	0.67
50	8740	72.03	56.14	1.52
75	12014	98.30	81.00	2.06

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	642	18.44	6.67	0.38
2	773	19.43	7.65	0.40
3	903	20.39	8.62	0.42
5	1167	22.33	10.57	0.46
10	1822	27.03	15.40	0.56
50	7063	67.55	54.74	1.40
75	10338	93.99	79.63	1.94

`Close` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	669	17.78	11.67	0.41
2	800	18.74	12.64	0.43
3	931	19.72	13.62	0.45
5	1194	21.67	15.58	0.49
50	7091	67.34	59.97	1.44
75	10366	92.94	84.69	1.97

`Contest` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	700	21.50	14.91	0.46
2	832	22.66	15.94	0.48
3	960	23.77	16.96	0.51
5	1226	26.05	19.01	0.55
10	1881	31.74	24.15	0.66
50	7122	79.52	65.73	1.58
66	9222	99.38	82.56	1.95

`FanOut` transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTXO for better comparability.
Rows first grow the UTxO set at a fixed 10 parties, then show the largest set that still fits per number of parties (burning more participation tokens leaves less room for outputs).

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
10	0	0	5644	23.29	42.87	0.90
10	1	57	5678	25.63	45.39	0.93
10	5	284	5814	35.88	55.71	1.10
10	10	569	5983	49.89	68.99	1.31
10	20	1138	6322	82.42	96.97	1.79
1	20	1140	6046	76.20	95.00	1.72
5	20	1137	6167	78.96	95.88	1.75
10	20	1136	6320	82.42	96.97	1.79
20	20	1139	6634	89.74	99.26	1.88
50	15	854	7395	94.70	91.88	1.90

`PartialFanOut` transaction costs

Largest chunk of ada-only outputs that can be distributed in one partial fanout step, computed dynamically. The last row is the maximum total UTxO count where at least one output can still be distributed.

Total UTxO	Distributed	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
11	10	570	983	34.91	66.33	0.95
25	23	1309	1424	68.24	99.40	1.48
30	23	1309	1428	68.24	99.40	1.48
40	23	1309	1428	68.24	99.40	1.48
50	23	1312	1431	68.24	99.40	1.48
100	23	1311	1430	68.24	99.40	1.48
150	23	1307	1426	68.24	99.40	1.48
200	23	1308	1427	68.24	99.40	1.48
200	23	1310	1429	68.24	99.40	1.48

`PartialFanOut` transaction costs (with native tokens)

Largest chunk of native-token outputs that can be distributed in one partial fanout step, computed dynamically. The last row is the maximum total UTxO count where at least one output can still be distributed.

Total UTxO	Distributed	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
11	10	930	1396	42.00	68.83	1.05
25	21	1995	2232	76.56	99.11	1.58
30	21	2394	2654	76.56	99.22	1.60
40	21	1995	2236	76.56	99.11	1.58
50	21	1974	2215	76.56	99.11	1.58
100	21	2415	2677	76.59	99.22	1.60
150	21	2016	2259	76.56	99.11	1.58
200	21	2268	2523	76.59	99.17	1.60
200	21	2142	2391	76.59	99.17	1.59

`FinalPartialFanOut` transaction costs (with native tokens)

Terminal partial fanout step (FanoutProgress → Final) with outputs carrying a native token. Burns all head tokens and proves accumulator exhaustion via BLS proof.

Distributed	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	93	5512	22.10	44.30	0.89
5	475	5810	35.58	55.79	1.10
10	1140	6371	53.78	70.62	1.37
10	1020	6250	53.78	70.59	1.37

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2026-08-13 15:52:11.232347258 UTC

Baseline Scenario

Number of nodes	1
Number of txs	300
Avg. Confirmation Time (ms)	194.5
P99	197.7ms
P95	197.5ms
P50	195.2ms
Tx validation time p50 (ms)	122.0
End-to-end TPS	1515.12 tx/s
Backlog drain time (s)	0.2
Snapshots observed	2
Snapshots per second	10.10 /s
Avg txs per snapshot	150.0
Peak node RSS (MB)	144.4
Number of Invalid txs	0
Fanout outputs	2

Three local nodes

Number of nodes	3
Number of txs	900
Avg. Confirmation Time (ms)	990.8
P99	1056.8ms
P95	1051.7ms
P50	998.0ms
Tx validation time p50 (ms)	480.1
End-to-end TPS	832.34 tx/s
Backlog drain time (s)	1.0
Snapshots observed	3
Snapshots per second	2.77 /s
Avg txs per snapshot	300.0
Peak node RSS (MB)	146.3
Number of Invalid txs	0
Fanout outputs	4

Scenario benchmark results

This page collects results from the scenario matrix: every combination of cluster size, UTxO shape, and incremental-ops mode is exercised by CI from the latest master code and reported below.

Numbers are approximate. They come from cloud VMs rather than controlled hardware, so the useful signal is the relative change between cells and between commits, not the absolute throughput.

Generated at 2026-08-13 16:05:30.217294246 UTC

Summary across cells

TPS columns are rates (transactions per second); Wall clock (s) is the measured elapsed time from the first tx submission to the last confirmation. Times are rounded to one decimal.

Scenario	Txs	Wall clock (s)	End-to-end TPS (tx/s)	Sustained TPS (tx/s)	Avg conf (ms)	P95 conf (ms)
Nodes=1, Constant, fire and forget	30	0.0	1279.77	n/a	22.6	23.2
Nodes=1, Constant, wait for tx valid	30	0.1	234.42	236.79	4.2	4.9
Nodes=1, Growing, fire and forget	30	0.0	852.42	n/a	34.4	34.9
Nodes=1, Growing, wait for tx valid	30	0.2	168.44	166.66	5.9	7.8
Nodes=1, Mixed, fire and forget	30	0.0	1046.13	n/a	28.0	28.4
Nodes=1, Mixed, wait for tx valid	30	0.2	187.44	182.48	5.3	7.1
Nodes=2, Constant, fire and forget	60	0.1	1121.78	n/a	51.8	52.8
Nodes=2, Constant, wait for tx valid	60	0.4	141.22	138.98	14.0	20.7
Nodes=2, Growing, fire and forget	60	0.1	890.87	n/a	65.8	66.6
Nodes=2, Growing, wait for tx valid	60	0.7	91.44	90.54	21.7	30.3
Nodes=2, Mixed, fire and forget	60	0.1	865.84	n/a	67.9	69.0
Nodes=2, Mixed, wait for tx valid	60	0.6	100.69	95.77	19.7	27.9
Nodes=3, Constant, fire and forget	90	0.1	916.79	n/a	94.7	97.8
Nodes=3, Constant, wait for tx valid	90	0.7	123.19	123.84	24.0	30.0
Nodes=3, Growing, fire and forget	90	0.2	563.74	n/a	155.1	157.8
Nodes=3, Growing, wait for tx valid	90	1.2	75.77	76.56	38.5	50.2
Nodes=3, Mixed, fire and forget	90	0.1	672.17	n/a	129.6	133.7
Nodes=3, Mixed, wait for tx valid	90	1.1	84.52	83.12	35.0	49.0

Nodes=1, Constant, fire and forget

Number of nodes	1
Number of txs	30
Avg. Confirmation Time (ms)	22.6
P99	23.2ms
P95	23.2ms
P50	22.7ms
Tx validation time p50 (ms)	10.1
End-to-end TPS	1279.77 tx/s
Backlog drain time (s)	0.0
Snapshots observed	2
Snapshots per second	85.32 /s
Avg txs per snapshot	15.0
Peak node RSS (MB)	145.0
Number of Invalid txs	0
Fanout outputs	2

Nodes=1, Constant, wait for tx valid

Number of nodes	1
Number of txs	30
Avg. Confirmation Time (ms)	4.2
P99	5.0ms
P95	4.9ms
P50	4.1ms
Tx validation time p50 (ms)	1.5
End-to-end TPS	234.42 tx/s
Sustained TPS	236.79 tx/s
Backlog drain time (s)	0.0
Snapshots observed	30
Snapshots per second	234.42 /s
Avg txs per snapshot	1.0
Peak node RSS (MB)	143.9
Number of Invalid txs	0
Fanout outputs	2

Nodes=1, Growing, fire and forget

Number of nodes	1
Number of txs	30
Avg. Confirmation Time (ms)	34.4
P99	35.0ms
P95	34.9ms
P50	34.7ms
Tx validation time p50 (ms)	17.7
End-to-end TPS	852.42 tx/s
Backlog drain time (s)	0.0
Snapshots observed	2
Snapshots per second	56.83 /s
Avg txs per snapshot	15.0
Peak node RSS (MB)	144.4
Number of Invalid txs	0
Fanout outputs	31

Nodes=1, Growing, wait for tx valid

Number of nodes	1
Number of txs	30
Avg. Confirmation Time (ms)	5.9
P99	8.3ms
P95	7.8ms
P50	5.7ms
Tx validation time p50 (ms)	1.6
End-to-end TPS	168.44 tx/s
Sustained TPS	166.66 tx/s
Backlog drain time (s)	0.0
Snapshots observed	30
Snapshots per second	168.44 /s
Avg txs per snapshot	1.0
Peak node RSS (MB)	144.4
Number of Invalid txs	0
Fanout outputs	31

Nodes=1, Mixed, fire and forget

Each client first grows its UTxO set (1-in to 2-out) for half of its tx budget, then contracts it back (2-in to 1-out) for the remainder.

Number of nodes	1
Number of txs	30
Avg. Confirmation Time (ms)	28.0
P99	28.4ms
P95	28.4ms
P50	28.1ms
Tx validation time p50 (ms)	13.9
End-to-end TPS	1046.13 tx/s
Backlog drain time (s)	0.0
Snapshots observed	2
Snapshots per second	69.74 /s
Avg txs per snapshot	15.0
Peak node RSS (MB)	143.1
Number of Invalid txs	0
Fanout outputs	2

Nodes=1, Mixed, wait for tx valid

Each client first grows its UTxO set (1-in to 2-out) for half of its tx budget, then contracts it back (2-in to 1-out) for the remainder.

Number of nodes	1
Number of txs	30
Avg. Confirmation Time (ms)	5.3
P99	8.6ms
P95	7.1ms
P50	5.1ms
Tx validation time p50 (ms)	1.5
End-to-end TPS	187.44 tx/s
Sustained TPS	182.48 tx/s
Backlog drain time (s)	0.0
Snapshots observed	30
Snapshots per second	187.44 /s
Avg txs per snapshot	1.0
Peak node RSS (MB)	144.1
Number of Invalid txs	0
Fanout outputs	2

Nodes=2, Constant, fire and forget

Number of nodes	2
Number of txs	60
Avg. Confirmation Time (ms)	51.8
P99	53.1ms
P95	52.8ms
P50	51.9ms
Tx validation time p50 (ms)	36.9
End-to-end TPS	1121.78 tx/s
Backlog drain time (s)	0.1
Snapshots observed	2
Snapshots per second	37.39 /s
Avg txs per snapshot	30.0
Peak node RSS (MB)	144.8
Number of Invalid txs	0
Fanout outputs	3

Nodes=2, Constant, wait for tx valid

Number of nodes	2
Number of txs	60
Avg. Confirmation Time (ms)	14.0
P99	21.6ms
P95	20.7ms
P50	13.2ms
Tx validation time p50 (ms)	4.2
End-to-end TPS	141.22 tx/s
Sustained TPS	138.98 tx/s
Backlog drain time (s)	0.0
Snapshots observed	60
Snapshots per second	141.22 /s
Avg txs per snapshot	1.0
Peak node RSS (MB)	145.2
Number of Invalid txs	0
Fanout outputs	3

Nodes=2, Growing, fire and forget

Number of nodes	2
Number of txs	60
Avg. Confirmation Time (ms)	65.8
P99	66.6ms
P95	66.6ms
P50	66.1ms
Tx validation time p50 (ms)	21.9
End-to-end TPS	890.87 tx/s
Backlog drain time (s)	0.1
Snapshots observed	2
Snapshots per second	29.70 /s
Avg txs per snapshot	30.0
Peak node RSS (MB)	144.7
Number of Invalid txs	0
Fanout outputs	62

Nodes=2, Growing, wait for tx valid

Number of nodes	2
Number of txs	60
Avg. Confirmation Time (ms)	21.7
P99	34.3ms
P95	30.3ms
P50	21.0ms
Tx validation time p50 (ms)	7.1
End-to-end TPS	91.44 tx/s
Sustained TPS	90.54 tx/s
Backlog drain time (s)	0.0
Snapshots observed	60
Snapshots per second	91.44 /s
Avg txs per snapshot	1.0
Peak node RSS (MB)	146.2
Number of Invalid txs	0
Fanout outputs	62

Nodes=2, Mixed, fire and forget

Each client first grows its UTxO set (1-in to 2-out) for half of its tx budget, then contracts it back (2-in to 1-out) for the remainder.

Number of nodes	2
Number of txs	60
Avg. Confirmation Time (ms)	67.9
P99	69.1ms
P95	69.0ms
P50	68.2ms
Tx validation time p50 (ms)	26.8
End-to-end TPS	865.84 tx/s
Backlog drain time (s)	0.1
Snapshots observed	2
Snapshots per second	28.86 /s
Avg txs per snapshot	30.0
Peak node RSS (MB)	144.2
Number of Invalid txs	0
Fanout outputs	3

Nodes=2, Mixed, wait for tx valid

Each client first grows its UTxO set (1-in to 2-out) for half of its tx budget, then contracts it back (2-in to 1-out) for the remainder.

Number of nodes	2
Number of txs	60
Avg. Confirmation Time (ms)	19.7
P99	31.5ms
P95	27.9ms
P50	19.4ms
Tx validation time p50 (ms)	5.8
End-to-end TPS	100.69 tx/s
Sustained TPS	95.77 tx/s
Backlog drain time (s)	0.0
Snapshots observed	60
Snapshots per second	100.69 /s
Avg txs per snapshot	1.0
Peak node RSS (MB)	144.6
Number of Invalid txs	0
Fanout outputs	3

Nodes=3, Constant, fire and forget

Number of nodes	3
Number of txs	90
Avg. Confirmation Time (ms)	94.7
P99	97.9ms
P95	97.8ms
P50	95.0ms
Tx validation time p50 (ms)	43.7
End-to-end TPS	916.79 tx/s
Backlog drain time (s)	0.1
Snapshots observed	2
Snapshots per second	20.37 /s
Avg txs per snapshot	45.0
Peak node RSS (MB)	145.1
Number of Invalid txs	0
Fanout outputs	4

Nodes=3, Constant, wait for tx valid

Number of nodes	3
Number of txs	90
Avg. Confirmation Time (ms)	24.0
P99	32.6ms
P95	30.0ms
P50	23.2ms
Tx validation time p50 (ms)	6.8
End-to-end TPS	123.19 tx/s
Sustained TPS	123.84 tx/s
Backlog drain time (s)	0.0
Snapshots observed	61
Snapshots per second	83.49 /s
Avg txs per snapshot	1.5
Peak node RSS (MB)	145.1
Number of Invalid txs	0
Fanout outputs	4

Nodes=3, Growing, fire and forget

Number of nodes	3
Number of txs	90
Avg. Confirmation Time (ms)	155.1
P99	157.9ms
P95	157.8ms
P50	155.0ms
Tx validation time p50 (ms)	86.1
End-to-end TPS	563.74 tx/s
Backlog drain time (s)	0.2
Snapshots observed	2
Snapshots per second	12.53 /s
Avg txs per snapshot	45.0
Peak node RSS (MB)	145.2
Number of Invalid txs	0
Fanout outputs	0

Nodes=3, Growing, wait for tx valid

Number of nodes	3
Number of txs	90
Avg. Confirmation Time (ms)	38.5
P99	62.4ms
P95	50.2ms
P50	38.2ms
Tx validation time p50 (ms)	11.0
End-to-end TPS	75.77 tx/s
Sustained TPS	76.56 tx/s
Backlog drain time (s)	0.0
Snapshots observed	63
Snapshots per second	53.04 /s
Avg txs per snapshot	1.4
Peak node RSS (MB)	146.9
Number of Invalid txs	0
Fanout outputs	0

Nodes=3, Mixed, fire and forget

Each client first grows its UTxO set (1-in to 2-out) for half of its tx budget, then contracts it back (2-in to 1-out) for the remainder.

Number of nodes	3
Number of txs	90
Avg. Confirmation Time (ms)	129.6
P99	133.8ms
P95	133.7ms
P50	129.2ms
Tx validation time p50 (ms)	56.9
End-to-end TPS	672.17 tx/s
Backlog drain time (s)	0.1
Snapshots observed	2
Snapshots per second	14.94 /s
Avg txs per snapshot	45.0
Peak node RSS (MB)	145.0
Number of Invalid txs	0
Fanout outputs	4

Nodes=3, Mixed, wait for tx valid

Each client first grows its UTxO set (1-in to 2-out) for half of its tx budget, then contracts it back (2-in to 1-out) for the remainder.

Number of nodes	3
Number of txs	90
Avg. Confirmation Time (ms)	35.0
P99	52.1ms
P95	49.0ms
P50	34.3ms
Tx validation time p50 (ms)	8.8
End-to-end TPS	84.52 tx/s
Sustained TPS	83.12 tx/s
Backlog drain time (s)	0.0
Snapshots observed	62
Snapshots per second	58.22 /s
Avg txs per snapshot	1.5
Peak node RSS (MB)	145.8
Number of Invalid txs	0
Fanout outputs	4

noonio · 2026-08-13T10:32:21Z


+-- * Orphans
+
+-- missing CBOR instances


out of interest is there a reason these instances should be missing? is there some reason upstream declined to provide them? could we open a PR for it somewhere?

noonio · 2026-08-13T10:37:48Z

+    SideLoadSnapshotFailed{sideLoadRequirementFailure} ->
+      toCBOR ("SideLoadSnapshotFailed" :: Text) <> toCBOR sideLoadRequirementFailure
+
+instance (IsChainState tx, FromCBOR (Input tx)) => FromCBOR (LogicError tx) where


Is there not a kind of generics-based cbor derivation we can do? So we don't need to do all these manually?

noonio

Looks good; I think the only issue I have is all the verbose instances you had to write "by hand"; I wonder, can we use this library - https://hackage.haskell.org/package/serialise - to get a generics-based instance instead? Claude seems to think so; but not sure if you tried it already and found problems?

Signed-off-by: Sasha Bogicevic <sasha.bogicevic@iohk.io>

Add ToCBOR/FromCBOR for StateChanged and StateEvent and store event rows as CBOR: 2.6x smaller on a real database and faster to replay. Existing databases are migrated automatically (schema v2, one-shot re-encode in a single transaction + VACUUM); a corrupt row aborts startup and leaves the v1 database intact. No downgrade path. The client API is unaffected — JSON output stays byte-identical. The on-disk format is locked by roundtrip properties (genStateChanged now covers all constructors) and a golden CBOR file with one sample per constructor. Signed-off-by: Sasha Bogicevic <sasha.bogicevic@iohk.io>

Signed-off-by: Sasha Bogicevic <sasha.bogicevic@iohk.io>

v0d1ch · 2026-08-13T13:55:38Z

There is no existing library for name-tagged generic CBOR: the ecosystem is cborg (manual), serialise (generic but constructor-index tags, not configurable) and cborg-json. serialise's own tutorial recommends manual instances when you need to manage migrations and reserves the right to change its derived encoding between major versions, which rules it out for hydra.db either way.
I will build quickly some in-house generics deriving solution to this.

noonio · 2026-08-13T14:30:23Z

There is no existing library for name-tagged generic CBOR: the ecosystem is cborg (manual), serialise (generic but constructor-index tags, not configurable) and cborg-json. serialise's own tutorial recommends manual instances when you need to manage migrations and reserves the right to change its derived encoding between major versions, which rules it out for hydra.db either way. I will build quickly some in-house generics deriving solution to this.

Ah, I see, and you want tags instead of indexes because you're thinking about being backwards-compatible between say adding new constructors?

and i suppose that would be fine here with the manual versions

yeah; i feel like maybe it could be fun to see if it's easy to fork serialise to do a tag-based scheme? maybe that's not too hard to add ?

Add Hydra.CBOR.Generic to hydra-prelude, providing genericToCBOR / genericFromCBOR (re-exported from Hydra.Prelude). The derived format is exactly the hand-written one: the constructor name as a CBOR text tag followed by the fields in declaration order, with no list framing. Every constructor is tagged, including newtypes and single-constructor records, and unknown tags fail with the same error text as before. Name-based tags keep persisted data decodable when constructors are added or reordered, unlike the index-based generic encoding of the serialise package. Swap 22 hand-written instance pairs to the deriver. Byte compatibility is locked by new per-constructor golden files (golden/*.cbor, via ToADTArbitrary): they were generated from the old hand-written encoders and must decode and re-encode identically with the derived ones. Deliberate format changes in this commit: NetworkInfo now carries its constructor tag like the other API types (Greetings bytes change transitively); both goldens regenerated. The golden enumeration also exposed that the hand-written ToCBOR (PostTxError tx) missed the DepositTooLarge constructor, crashing with PatternMatchFail on encode; the derived instance covers it by construction. Also add a typos.toml exception: the typos formatter rewrote GHC.TypeLits into the nonexistent GHC.TypeList on every nix fmt. Signed-off-by: Sasha Bogicevic <sasha.bogicevic@iohk.io>

v0d1ch · 2026-08-13T15:42:15Z

Take a look at the last commit - we now have our own generic deriving but we still need to keep some types out of it since they had different encodings. If you like it we can keep it or default to manual instances or do some other experiment like fork serialise.

v0d1ch self-assigned this Jul 20, 2026

v0d1ch force-pushed the binary-db branch 2 times, most recently from 23e1891 to 8310dea Compare August 3, 2026 16:29

v0d1ch changed the title ~~Switch to cbor format in DB~~ CBOR-1: Switch to cbor format in DB Aug 4, 2026

v0d1ch force-pushed the binary-db branch 2 times, most recently from aa133a6 to a9dbc15 Compare August 11, 2026 13:54

noonio added this to the Performance milestone Aug 13, 2026

noonio reviewed Aug 13, 2026

View reviewed changes

Comment thread head-state-viewer/src/HydraVis/UI/Update.hs

noonio reviewed Aug 13, 2026

View reviewed changes

Comment thread head-state-viewer/src/HydraVis/History.hs

noonio reviewed Aug 13, 2026

View reviewed changes

Comment thread hydra-node/golden/StateEvent.cbor

noonio reviewed Aug 13, 2026

View reviewed changes

Comment thread hydra-node/test/Hydra/CBORSpec.hs

noonio requested changes Aug 13, 2026

View reviewed changes

v0d1ch added 3 commits August 13, 2026 15:19

Add CBOR instances

4e94ab3

Signed-off-by: Sasha Bogicevic <sasha.bogicevic@iohk.io>

Further enforce the migration

c4925c8

Signed-off-by: Sasha Bogicevic <sasha.bogicevic@iohk.io>

v0d1ch force-pushed the binary-db branch from a9dbc15 to c4925c8 Compare August 13, 2026 13:21

Navigation Menu

Uh oh!

CBOR-1: Switch to cbor format in DB - #2767

CBOR-1: Switch to cbor format in DB#2767
v0d1ch wants to merge 4 commits into
masterfrom
binary-db

v0d1ch commented Jul 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 20, 2026

Uh oh!

github-actions Bot commented Jul 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

noonio Aug 13, 2026

Uh oh!

Uh oh!

noonio Aug 13, 2026

Uh oh!

Uh oh!

noonio left a comment

Uh oh!

v0d1ch commented Aug 13, 2026

Uh oh!

noonio commented Aug 13, 2026

Uh oh!

v0d1ch commented Aug 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

v0d1ch commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Compatibility

Format stability in CI

Uh oh!

github-actions Bot commented Jul 20, 2026

Transaction cost differences

Uh oh!

github-actions Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

End-to-end benchmark differences

Sustained load (3 nodes, 3x5000 txs)

Plateau 1000 UTxO (1 node, 4000 txs)

Round-trip latency (3 nodes, closed-loop, 3x250 txs)

Uh oh!

github-actions Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Transaction costs

Script summary

Init transaction costs

Cost of Increment Transaction

Cost of Decrement Transaction

Close transaction costs

Contest transaction costs

FanOut transaction costs

PartialFanOut transaction costs

PartialFanOut transaction costs (with native tokens)

FinalPartialFanOut transaction costs (with native tokens)

End-to-end benchmark results

Baseline Scenario

Three local nodes

Scenario benchmark results

Summary across cells

Nodes=1, Constant, fire and forget

Nodes=1, Constant, wait for tx valid

Nodes=1, Growing, fire and forget

Nodes=1, Growing, wait for tx valid

Nodes=1, Mixed, fire and forget

Nodes=1, Mixed, wait for tx valid

Nodes=2, Constant, fire and forget

Nodes=2, Constant, wait for tx valid

Nodes=2, Growing, fire and forget

Nodes=2, Growing, wait for tx valid

Nodes=2, Mixed, fire and forget

Nodes=2, Mixed, wait for tx valid

Nodes=3, Constant, fire and forget

Nodes=3, Constant, wait for tx valid

Nodes=3, Growing, fire and forget

Nodes=3, Growing, wait for tx valid

Nodes=3, Mixed, fire and forget

Nodes=3, Mixed, wait for tx valid

Uh oh!

Uh oh!

Uh oh!

noonio Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

noonio Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

noonio left a comment

Choose a reason for hiding this comment

Uh oh!

v0d1ch commented Aug 13, 2026

Uh oh!

noonio commented Aug 13, 2026

Uh oh!

v0d1ch commented Aug 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

v0d1ch commented Jul 20, 2026 •

edited

Loading

github-actions Bot commented Jul 20, 2026 •

edited

Loading

github-actions Bot commented Jul 20, 2026 •

edited

Loading

`Init` transaction costs

`Close` transaction costs

`Contest` transaction costs

`FanOut` transaction costs

`PartialFanOut` transaction costs

`PartialFanOut` transaction costs (with native tokens)

`FinalPartialFanOut` transaction costs (with native tokens)