Skip to content

Commit 10d161b

Browse files
ryan-williamsclaude
andcommitted
RG manifest: widen eligibility to vocab-term covers (regions, bboxes)
`include` lists on rides-v5 are always vocab terms (`s:` passthrough or `vocabCover` translation), and vocab-pyramid avail covers are exact-match terms too — so the per-token manifest predicates are valid for every shape, not just `s:`-key covers. Measured region covers are ≤16 tokens (NYC = 16 vocab cells, 0 station keys), inside the 45-token predicate cap; dim-filtered rides queries and avail excludes keep the legacy guarded path. Verified byte-equal to the footer path on all three region ± covers (rides) and a mixed cells+`s:` cover (avail-v6), 1.4-3.2× faster cold. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 5007530 commit 10d161b

2 files changed

Lines changed: 21 additions & 9 deletions

File tree

gbfs/api/src/avail_geo.ts

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -600,9 +600,18 @@ async function serveGeoReduced(
600600
// back to the footer path per key and lazily fill (big shards only —
601601
// see `MIN_FILL_RGS`; the Lambda-churned tip shards stay on the cheap
602602
// fallback so their constant rewrites don't grind D1 writes).
603+
// Manifest fast path (`specs/rg-manifest.md` P3): explicit covers on
604+
// vocab pyramids use exact-match per-token predicates — valid for any
605+
// vocab term (`s:` keys AND vocab cells; rows exist only at vocab
606+
// terms, so a non-vocab token matches nothing on either path).
607+
// Region-scale vocab covers measure ≤16 tokens (NYC), well inside the
608+
// predicate builder's 45-token cap. Excludes stay on the legacy path
609+
// (query-time subtraction; rare). Non-vocab pyramids (avail-v3) keep
610+
// the legacy path: their S2 cover cells match rows by containment,
611+
// not equality, so per-token exact predicates would undercount.
603612
const manifestEligible = userCells !== null
604613
&& userCellsExclude.length === 0
605-
&& userCells.every((c) => c.startsWith('s:'));
614+
&& (PYRAMIDS[pyramidName]?.vocab === true || userCells.every((c) => c.startsWith('s:')));
606615
// Thread `binCol` + per-segment range so hyparquet prunes row groups by
607616
// `dt` column stats — shards are `(dt, s2_cell)`-sorted with small
608617
// row groups, so a sub-shard time window reads only the matching RGs.

gbfs/api/src/rides_v1.ts

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -741,14 +741,17 @@ export async function serveRidesV5(
741741
registered,
742742
);
743743
const rgFilters = parseDimFilters(url) ?? [];
744-
// RG-manifest path (`specs/rg-manifest.md` P1): `s:`-key covers with no
745-
// dim filters serve from the D1 row-group index — no footer parse, so
746-
// segments fan out in parallel and the footer guard doesn't apply
747-
// (misses fall back to the guarded footer path per key, and fill the
748-
// manifest via `defer`). Dim-filtered or vocab-cell queries keep the
749-
// legacy guarded sequential path (dim RG-prune semantics + large
750-
// token sets are out of P1 scope).
751-
const manifestEligible = rgFilters.length === 0 && include.every((c) => c.startsWith('s:'));
744+
// RG-manifest path (`specs/rg-manifest.md` P1+P3): serve from the D1
745+
// row-group index — no footer parse, so segments fan out in parallel
746+
// and the footer guard doesn't apply (misses fall back to the guarded
747+
// footer path per key, and fill the manifest via `defer`). `include`
748+
// is always a vocab term list here (`s:` passthrough, or raw
749+
// covers/bboxes translated positive-only via `vocabCover`), so
750+
// per-token exact-match predicates are valid for every shape;
751+
// region-scale covers measure ≤16 tokens (NYC), inside the predicate
752+
// builder's 45-token cap. Dim-filtered queries keep the legacy
753+
// guarded sequential path (dim RG-prune semantics are out of scope).
754+
const manifestEligible = rgFilters.length === 0;
752755
let shardRows: Row[][];
753756
try {
754757
if (manifestEligible) {

0 commit comments

Comments
 (0)