Skip to content

feat: Open Data API — public read-only API with keys and quotas (#1060) - #1095

Merged
Anuoluwapo25 merged 4 commits into
bakeronchain:mainfrom
thebabalola:feat/issue-1060-open-data-api
Aug 27, 2026
Merged

feat: Open Data API — public read-only API with keys and quotas (#1060)#1095
Anuoluwapo25 merged 4 commits into
bakeronchain:mainfrom
thebabalola:feat/issue-1060-open-data-api

Conversation

@thebabalola

Copy link
Copy Markdown
Contributor

Closes #1060

Summary of Changes

Adds a public, read-only, API-key-authenticated surface under /api/v1/public so third parties can self-serve LearnVault data: course catalog, aggregate platform stats, learner leaderboard, treasury flows and credential verification. Keys are shown once at creation and stored only as SHA-256 hashes; per-tier daily quotas keep the endpoint honest. Privacy is treated as the core of the issue — k-anonymity suppression on every aggregate, explicit safe-column selection on identity-adjacent queries.

What Changed

  • Migration 034 (api_keys + api_key_usage): hash-only key storage, usage tracked per key per endpoint per day. (The issue sketched this as migration 030, but 030–033 were already taken in this repo — numbered 034 instead.)
  • Key middleware (X-API-Key): missing/malformed/unknown/revoked keys all get 401; usage is recorded per endpoint per day; tier quotas (free = 1,000/day, partner = 10,000/day) return 429 with a Retry-After header counting down to UTC midnight. Quota headers ship on every response.
  • Five endpoints, versioned from the first commit:
    • GET /courses — published catalog, page/limit pagination
    • GET /stats — active learners, completions, LRN distributed, scholarships funded
    • GET /leaderboard — ranked public handles only; raw wallet addresses never leave
    • GET /treasury — daily inflow/outflow aggregates over a period
    • GET /credentials/:id/verify — validity + course slug + issue date, no holder identity
  • Privacy by construction: every aggregate suppresses buckets with fewer than 5 subjects, and the leaderboard controller selects explicit safe fields rather than spreading rows — my privacy test actually caught that gap mid-build (a careless spread would have leaked whatever columns a query returned), so the hardening is in place even if queries change later.
  • OpenAPI: all five routes documented with the new ApiKeyAuth security scheme.

Testing / Local Verification

12 new tests in open-data.routes.test.ts:

  • Missing header, malformed prefix, unknown key and revoked key each rejected with 401 (malformed rejected before any DB hit)
  • Quota boundary returns 429 with Retry-After > 0; next day resets cleanly
  • Usage upsert keyed on (key_id, endpoint, day)
  • Small-bucket suppression on stats and full leaderboard suppression under 5 learners
  • Pagination across three pages with no duplicates or gaps
  • Identity-leak scan: responses asserted to contain no email-shaped strings, no kyc columns, no raw addresses — with mock rows deliberately carrying those columns to prove the controllers don't select them

Full server suite: 654 passing vs 642 at base; same 16 pre-existing failures as base, untouched.

api_keys stores only SHA-256 hashes — plaintext shown once at creation.
Usage tracked per key per endpoint per day for tier quotas (free 1000/day,
partner 10k/day).
…ain#1060)

- X-API-Key middleware: rejects missing/malformed/revoked keys with 401,
  records usage per endpoint per day, enforces tier quota with 429 +
  Retry-After until UTC midnight
- five read-only endpoints: course catalog, aggregate stats, leaderboard,
  treasury flows, credential verification — all versioned from day one
- privacy by construction: k>=5 bucket suppression on every aggregate,
  leaderboard exposes handles only (raw wallet addresses hashed out),
  controllers select explicit safe columns so identity columns can't ride
  along even if a query changes later
All five /v1/public routes with ApiKeyAuth security scheme.
12 tests: missing/malformed/unknown/revoked keys, quota boundary with
429 + Retry-After and next-day reset, per-endpoint usage counters,
k<5 bucket suppression on stats and leaderboard, pagination stability,
and an identity-leak scan asserting no email/kyc/raw-address shapes in
any response body across endpoints.
@thebabalola

Copy link
Copy Markdown
Contributor Author

@Anuoluwapo25 PR is ready for your review. It closes #1060 ; five public read-only endpoints under a versioned /api/v1/public path, API keys stored hash-only with tiered daily quotas, and privacy handled as the core requirement (k>=5 suppression everywhere, no identity columns ever selected).
Two notes in the description: migration numbered 034 since 030 was taken, and one design call where the leaderboard controller picks explicit fields so identity columns can't leak even if queries change later. Happy to adjust either.

@Anuoluwapo25
Anuoluwapo25 merged commit 825c359 into bakeronchain:main Aug 27, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: LearnVault Open Data API — public read-only API with keys and quotas

2 participants