docs: record the secondary read-view contract and API-016

The read-view pinning added in 3724c8a was documented nowhere, and it
constrains every new API read path. The 06f3fe9 check-count bump missed one
call site.

STORE_SCHEMA.md
- New "Read Consistency (secondary readers)" section: a secondary cannot take
  snapshots and its view advances only on `try_catch_up_with_primary()`, so
  `read_view.rs` makes catch-up exclusive with pinned read scopes. Covers the
  `CatchUpWindow` compile-time rule, one pinned view per HTTP request, the
  three deliberately unpinned readers (cycles long-poll, cache warmup, WS
  broadcasters), both bounds (100ms yield, 5s stall log), and the self-deadlock
  rule.

ARCHITECTURE_MAP.md
- Data-flow step 4 names the catch-up loop, Store Ownership gains the rule a
  new read path has to know, and the Store row lists `read_view.rs`.

CLAUDE.md
- The store section carries the pin rule, including that a handler which waits
  for new data must release its pin first; the Gotchas row no longer stops at
  "read-only, no write locks".

POSTMORTEM.md
- API-016: the 500 on `GET /dao/deposits?status=0`. The index was exact
  (91,508/91,508 mainnet, 9,288/9,288 testnet); what had been lost was the
  snapshot the assertion was written against — added in ca9afe0, dropped again
  in 559e05f to make the path run on a secondary, with a regression test that
  only exercised the primary branch.

NETWORK_PEER_CRAWLER.md
- 56 -> 57 checks, the one site 06f3fe9 missed.

Re-sync required: no.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JsRbGi6hngQBnJXm6GjB3g

Author: janx

CLAUDE.md

@@ -191,7 +191,7 @@ Sync progress and memory stats are stored in RocksDB (`get_sync_tip()`/`get_sync
 
 ## ckbadger-store (Embedded Storage Engine)
 
-Three logical RocksDB store classes: domain (`[store].domain_data_path`, default `data/domain`, 59 CFs), append-only (`[store].append_only_data_path`, default `data/append-only`, 1 CF: `CF_CELLS`), and network (`[store].network_data_path`, default `data/network`, 2 CFs: `CF_NET_NODES`, `CF_NET_STATS`). For the two chain stores the indexer opens read-write and the API opens secondary (read-only); the append-only store holds only immutable cell payloads keyed by outpoint, while all other chain state (activities, indexes, stats, etc.) lives in the domain store. The network store is written solely by the opt-in `ckbadger-crawler` service (configured via the `[crawler]` section, default `enabled = false`; API opens it secondary) and holds non-chain p2p-crawler observations — it is the only store EXEMPT from rebuild-from-genesis. See `docs/STORE_SCHEMA.md` for full column family reference.
+Three logical RocksDB store classes: domain (`[store].domain_data_path`, default `data/domain`, 59 CFs), append-only (`[store].append_only_data_path`, default `data/append-only`, 1 CF: `CF_CELLS`), and network (`[store].network_data_path`, default `data/network`, 2 CFs: `CF_NET_NODES`, `CF_NET_STATS`). For the two chain stores the indexer opens read-write and the API opens secondary (read-only); the append-only store holds only immutable cell payloads keyed by outpoint, while all other chain state (activities, indexes, stats, etc.) lives in the domain store. A secondary has no snapshots and its view advances only on `try_catch_up_with_primary()`, so the API pins one read view per HTTP request (`crates/ckbadger-store/src/read_view.rs`) and catch-up is exclusive with pinned scopes — any read that resolves an index row and then loads the row it points at is coherent by default. Handlers that wait for the indexer to write new data (the cycles long-poll) must release the pin first; see `docs/STORE_SCHEMA.md` → Read Consistency. The network store is written solely by the opt-in `ckbadger-crawler` service (configured via the `[crawler]` section, default `enabled = false`; API opens it secondary) and holds non-chain p2p-crawler observations — it is the only store EXEMPT from rebuild-from-genesis. See `docs/STORE_SCHEMA.md` for full column family reference.
 
 Memory is budgeted per network. `[store].memory_budget_gb` is an explicit per-network override;
 otherwise detected host RAM is divided by the number of co-resident orchestrator networks. The
@@ -342,7 +342,7 @@ const DAO_CODE_HASH: &str = "0x82d76d1b75fe2fd9a27dfbaa65a039221a380d76c926f378d
 | Vite SPA deep links       | Rust frontend server must fall back to `index.html` for non-files |
 | Vitest globals            | Add `vitest/globals` to tsconfig types                            |
 | MSW handlers              | Must start server in setup.ts `beforeAll`                         |
-| RocksDB secondary mode    | API uses `open_secondary()` — read-only, no write locks           |
+| RocksDB secondary mode    | Read-only, no write locks, no snapshots — pin a read view         |
 | Spore molecule `Bytes`    | Size field = content length (NOT total size including header)     |
 
 ## File Locations

docs/ARCHITECTURE_MAP.md

@@ -12,8 +12,8 @@ Quick navigation map for humans and agents working in `ckbadger`.
 3. Each indexer validates its configured network against the CKB node, derives the exact
    `GenesisBaseline` from block 0 when absent, fetches chain data, parses it, and writes its own
    domain and append-only RocksDB stores.
-4. Each API opens those two chain stores as read-only secondaries. A direct per-network API serves
-   `/api/v1/*` and `/ws`.
+4. Each API opens those two chain stores as read-only secondaries, refreshing them on a catch-up
+   loop. A direct per-network API serves `/api/v1/*` and `/ws`.
 5. The shared frontend serves pages under `/{network}/...` and proxies
    `/api/{network}/v1/*` → that network's `/api/v1/*`, plus
    `/ws/{network}` → that network's `/ws`.
@@ -28,14 +28,21 @@ Quick navigation map for humans and agents working in `ckbadger`.
 | Append-only | Per-network indexer | API secondary reader      | Immutable cell payloads keyed by outpoint                                 |
 | Network     | Per-network crawler | API secondary reader      | Non-chain p2p observations; TTL-retained and not rebuildable from genesis |
 
+A RocksDB secondary has no snapshots, so a reader's view changes only when catch-up runs.
+`crates/ckbadger-store/src/read_view.rs` makes catch-up exclusive with pinned read scopes, and the
+API pins one view per request: any new read path that resolves an index row and then loads the row
+it points at is coherent by default, and a handler that waits for the indexer to write new data
+(long-poll) must release its pin first. See
+[docs/STORE_SCHEMA.md](STORE_SCHEMA.md#read-consistency-secondary-readers).
+
 ## Module Map
 
 | Layer                    | Entry points                                              | Core files                                                                                             | Tests                                                     |
 | ------------------------ | --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | --------------------------------------------------------- |
 | CLI / supervisor         | `crates/cli/src/main.rs`                                  | `supervisor.rs`, `sequencer.rs`                                                                        | `cargo test -p ckbadger`                                  |
 | Configuration            | `crates/config/src/lib.rs`                                | `orchestrator.rs`                                                                                      | `cargo test -p ckbadger-config`                           |
 | Indexer library          | `crates/indexer/src/lib.rs`, `entry.rs`                   | `network_guard.rs`, `genesis_baseline.rs`, `lifecycle.rs`, `sync/`, `parser/`, `db/writer/`, `verify/` | `cargo test -p ckbadger-indexer`                          |
-| Store                    | `crates/ckbadger-store/src/lib.rs`                        | `store.rs`, `types.rs`, `keys.rs`, `*_ops.rs`, `network_*`                                             | `cargo test -p ckbadger-store`                            |
+| Store                    | `crates/ckbadger-store/src/lib.rs`                        | `store.rs`, `read_view.rs`, `types.rs`, `keys.rs`, `*_ops.rs`, `network_*`                             | `cargo test -p ckbadger-store`                            |
 | API / frontend server    | `crates/api/src/lib.rs`, `entry.rs`                       | `routes/`, `ws/`, `frontend_proxy.rs`, `embedded_frontend.rs`, `response.rs`                           | `cargo test -p ckbadger-api`, `crates/api/tests/api_*.rs` |
 | Crawler                  | `crates/crawler/src/lib.rs`                               | round engine, p2p prober, network-store writer                                                         | `cargo test -p ckbadger-crawler`                          |
 | TUI                      | `crates/tui/src/lib.rs`                                   | `multi.rs`, service/sync/memory/network panels                                                         | `cargo test -p ckbadger-tui`                              |

docs/NETWORK_PEER_CRAWLER.md

@@ -46,7 +46,7 @@ trends, and sampled peer gossip. The current UI does not render a topology graph
   (`get_peers` is local-node-only). Node records retain only a bounded sample of advertised peers;
   the current API and UI do not turn that sample into a topology graph.
 - Not a per-node detail page. A point API endpoint exists for hover popovers and API/agent use.
-- Not part of the chain data-integrity `verify` suite (observational data, outside the 56 checks).
+- Not part of the chain data-integrity `verify` suite (observational data, outside the 57 checks).
 
 ---
 

docs/POSTMORTEM.md

@@ -2358,4 +2358,61 @@ compute the answer that another path in the same process already has".
 
 ---
 
-_Last updated: 2026-08-09_
+### API-016: A fail-fast assertion whose premise the read path had lost
+
+**Date**: 2026-08-19
+
+**Symptom**: `GET /dao/deposits?status=0` intermittently returned 500 on healthy
+data, on mainnet and testnet:
+
+    dao_by_status_block stale status: expected=0, actual=1
+
+The index was not stale. A full-table audit of both live stores matched exactly
+(91,508 deposits vs 91,508 index rows on mainnet, 9,288 vs 9,288 on testnet),
+and the 0->1 transition it tripped on is an ordinary withdraw request.
+
+**Root Cause**: A RocksDB secondary cannot take snapshots (`GetSnapshot` fails
+with "snapshot not supported in secondary mode"), and its view advances _only_
+when `try_catch_up_with_primary()` runs — every second in the API. The listing
+resolves an index row and then loads the deposit it points at, so when a
+catch-up landed between the two, the iterator (pinned at creation) still yielded
+the pre-catch-up index row while the point lookup already saw the post-catch-up
+entry. The cross-check between them then failed on a perfectly normal
+transition. Reads without such a cross-check returned a torn mix of two views
+and said nothing.
+
+The snapshot-consistent version of this read existed: ca9afe04 added it, and
+559e05fb dropped the snapshot again to make the path work on a secondary — on
+the only path the API actually takes. Its regression test exercised the primary
+branch only, so CI stayed green over the broken one. The same contract had three
+copies of its checks, which is why a fix could live on one branch while the test
+covered another.
+
+**Fix**: `ckbadger_store::read_view` makes catch-up — the single mutation point
+of a reader's view — exclusive with pinned read scopes, restoring per process
+the guarantee `snapshot()` gives on a primary, which every multi-CF read path
+already assumes. `refresh()` runs inside a `CatchUpWindow` and
+`catch_up_in_window()` takes that window by reference, so "all secondaries
+advance together" is checked at compile time; the API pins one view per request
+in an innermost middleware, so a response cannot mix two views; the cycles
+long-poll, whose contract is to observe the _next_ view, releases its pin before
+waiting. Both sides are bounded (a scope arriving after a queued catch-up yields
+100ms, then pins anyway; a blocked catch-up logs the stall every 5s), so a
+request stream cannot starve catch-up and one wedged handler cannot freeze later
+requests. The "refresh append-only before domain" ordering workaround was
+deleted — one window now covers every secondary. One `DaoReadView` and one
+resolver replaced the three copies of the index/entry contract, and verify
+gained `dao_status_index_matches_deposits` (sampling), a full-table cross-check
+that reports INCONCLUSIVE on a reorg rather than blaming the index.
+
+**Lesson**: The assertion was right and the data was right; what had been
+quietly removed was the premise the assertion was written against. When a change
+drops a consistency primitive to make code run in a different mode, restore the
+primitive in that mode — do not leave invariant checks standing on a premise
+that no longer holds, and never silence them instead. A test that covers only
+the primary branch of a primary/secondary split covers the branch production
+does not take.
+
+---
+
+_Last updated: 2026-08-23_

docs/STORE_SCHEMA.md

@@ -191,6 +191,32 @@ Key = raw `peer_id` bytes → `NodeRecord` (per-peer crawler view: `own_addrs`,
 - `CkbadgerStore::open_domain_secondary(primary_path, secondary_path)` / `open_append_only_secondary(primary_path, secondary_path)` — read-only mode for API/TUI (split secondary stores)
 - All store operations are synchronous (RocksDB reads are fast)
 
+## Read Consistency (secondary readers)
+
+A secondary cannot take snapshots (`GetSnapshot` fails with "snapshot not supported in secondary
+mode"), and its view advances _only_ when `try_catch_up_with_primary()` runs. Without a read view,
+any read that resolves an index row and then loads the row it points at spans two views when a
+catch-up lands in between: the iterator, pinned at creation, still yields the pre-catch-up index
+row while the point lookup already sees the post-catch-up entry.
+
+`crates/ckbadger-store/src/read_view.rs` makes catch-up — the single mutation point of a reader's
+view — exclusive with pinned read scopes, restoring per process the guarantee `snapshot()` gives on
+a primary, which every multi-CF read path already assumes:
+
+- `CkbadgerStore::refresh` runs inside a `CatchUpWindow`; `catch_up_in_window()` takes that window
+  by reference, so "all secondaries advance together" is checked at compile time. Refresh order
+  between the domain and append-only stores is therefore invisible to readers.
+- The API pins one read view per HTTP request (innermost middleware), so a response can never mix
+  two views.
+- Deliberately **not** pinned: handlers that wait for the indexer to write new data (the cycles
+  long-poll releases its pin before waiting — its contract is to observe the _next_ view), plus
+  background full-store scans (cache warmup) and WebSocket broadcasters, which interleave external
+  I/O with minutes-long scans and accept drift rather than freeze catch-up.
+- Both sides are bounded: a read scope arriving after a catch-up queues yields to it for 100ms and
+  then pins anyway, and a catch-up held up by a read scope logs the stall every 5s.
+- Never hold a read view across `CkbadgerStore::refresh` on the same thread — that self-deadlocks.
+  Catch-up runs on its own thread (`spawn_blocking` in the API), never inside a pinned scope.
+
 ## Memory Considerations
 
 Memory sizing is per network rather than a fixed host-wide peak: