Skip to content

feat(core): parse Retry-After / Ratelimit headers and honor them in retry policy#228

Merged
sam-goodwin merged 8 commits into
mainfrom
feat/retry-after-headers
May 4, 2026
Merged

feat(core): parse Retry-After / Ratelimit headers and honor them in retry policy#228
sam-goodwin merged 8 commits into
mainfrom
feat/retry-after-headers

Conversation

@sam-goodwin

Copy link
Copy Markdown
Collaborator

Summary

Standardizes how SDKs surface server-provided retry hints. Stacks on top of #225.

  • Core defines an opaque retryAfter: Duration.Duration field on every retryable error (TooManyRequests, Locked, InternalServerError, BadGateway, ServiceUnavailable, GatewayTimeout).
  • Core ships parsing primitives: parseRetryAfter (RFC 7231 §7.1.3 — delay-seconds or HTTP-date), parseRatelimit (IETF httpapi-ratelimit-headers — r=0, t=30), and parseServerRetryHint (try one then the other).
  • Each SDK's matchError now accepts an optional headers arg and is responsible for deriving a Duration from whatever the service actually returns. Cloudflare, GCP, Kubernetes, plus all 15 scaffolded SDKs (turso, planetscale, neon, stripe, axiom, posthog, supabase, coinbase, fly-io, mongodb-atlas, typesense, workos, prisma-postgres, azure, expo-eas) populate retryAfter via parseServerRetryHint(headers) for the standard cases. The scripts/create-sdk.ts template emits the same wiring so newly scaffolded SDKs honor server hints by default.
  • Default retry policy (makeDefault) and the new throttlingFactory / transientFactory honor error.retryAfter with precedence over the exponential backoff, capped at 60s so a misbehaving server can't park us forever.
  • AWS untouched — uses its own retry stack.
  • Pipeline awareness: prompts in scripts/error-discovery.ts and scripts/generate-tests.ts teach sub-agents to preserve the headers parameter on matchError and pass retryAfter into retryable error constructors when patching, so the convention sticks across regeneration.

Override seam

If a service uses bespoke headers or body fields for cooldown, edit the SDK's matchError:

const retryAfter =
  parseServerRetryHint(headers) ??
  parseFooBespokeHeader(headers["x-foo-cooldown"]) ??
  parseFooBodyField(errorBody);
return Effect.fail(new TooManyRequests({ message, retryAfter }));

Test plan

  • bun run test in packages/core — 29 new unit tests pass (header parsing edge cases + policy honoring retryAfter)
  • bunx tsgo --noEmit clean for cloudflare, kubernetes, turso, planetscale, neon, stripe, axiom, posthog, supabase, coinbase, fly-io, mongodb-atlas, typesense, workos, prisma-postgres, azure, expo-eas
  • bunx oxlint src clean across all 18 SDKs
  • GCP shows pre-existing readonly-array errors only (also present on main, out of scope)

Made with Cursor

sam-goodwin and others added 5 commits May 1, 2026 19:08
The shared core `makeAPI` only ran a hard-coded throttling-only retry,
so Cloudflare's exported `Retry` Context.Service (`policy`, `none`,
`throttling`, `transient`) was effectively dead code. Callers had to
wrap individual operations with `Effect.retry({ ... })` instead of
installing one blanket policy at the layer level (the pattern AWS has
in `packages/aws/src/client/api.ts`).

Add an optional `retry?: Context.Key<any, RetryPolicy>` field to
`ClientConfig`. When set, every operation reads the policy via
`Effect.serviceOption(retry)` with `Retry.makeDefault` as the fallback
and a per-call `lastError` Ref, mirroring AWS's `outerFn`. When unset,
the legacy throttling-only schedule still runs, so other consumers
(turso, planetscale, neon, stripe, kubernetes, etc.) are unchanged
until they opt in.

Wire `retry: Retry` into Cloudflare's `makeAPI` so callers can do
`myEffect.pipe(Cloudflare.Retry.transient)` once and have every
Cloudflare API call below it transparently retry transient errors.

Co-authored-by: Cursor <cursoragent@cursor.com>
Per-SDK Retry tags were never actually consumed: each
`packages/<sdk>/src/retry.ts` declared its own `Retry extends
Context.Service<...>("FooRetry")`, but the shared `makeAPI` in
`@distilled.cloud/core/client` had no way to read from N different tags
and so just ran a hard-coded throttling-only schedule. The previous
commit on this branch threaded a per-SDK tag through `makeAPI` config —
that worked but it left every other SDK on the legacy throttling-only
path until they migrated.

Drop the per-SDK indirection entirely:

- `packages/core/src/retry.ts` now exports a single shared `Retry`
  Context.Service plus `policy`, `none`, `throttling`, `transient`
  helpers bound to it. `makeRetryService` is kept as a deprecated
  no-op shim for any external callers.
- `packages/core/src/client.ts`'s `makeAPI` always reads the shared
  `Retry` with `Effect.serviceOption`, falling back to
  `Retry.makeDefault` (transient/throttling/server with capped
  exponential backoff + jitter, 5 attempts). The `config.retry` field
  introduced in the previous commit and the legacy hard-coded
  throttling-only branch are removed. This mirrors AWS's pattern in
  `packages/aws/src/client/api.ts`.
- All 17 per-SDK `retry.ts` files (cloudflare, turso, planetscale,
  neon, stripe, kubernetes, axiom, posthog, gcp, coinbase, supabase,
  expo-eas, fly-io, mongodb-atlas, typesense, workos, prisma-postgres,
  azure) become thin re-exports of the shared core module so existing
  call sites (`Cloudflare.Retry.transient(eff)`,
  `Layer.succeed(Turso.Retry.Retry, customPolicy)`, etc.) keep working
  but now actually take effect.

AWS continues to use its own client and its own `Retry` tag —
unchanged in this PR.

Co-authored-by: Cursor <cursoragent@cursor.com>
…client

Reverts the shared-`Retry` approach from the previous commit. Each SDK
keeps its own `Retry` Context.Service tag (so policies don't bleed
across SDKs — `Cloudflare.Retry.transient` only retries Cloudflare
calls, not turso underneath), but each SDK's `client.ts` now wires its
own tag into `makeAPI` automatically. End users just write
`myEffect.pipe(Cloudflare.Retry.transient)` — no per-SDK config to
remember.

- `packages/core/src/client.ts`: `ClientConfig.retry` is now required.
  `makeAPI` always reads the configured tag with `Effect.serviceOption`
  and falls back to `Retry.makeDefault`. Removed the legacy
  throttling-only branch.
- `packages/core/src/retry.ts`: shared `Retry` tag/`policy`/`none`/
  `throttling`/`transient` removed; `makeRetryService` restored as the
  primary API for SDKs.
- 18 per-SDK `retry.ts` files: each declares its own `Retry` tag plus
  matching `policy`, `none`, `throttling`, `transient` helpers (added
  `throttling`/`transient` everywhere for parity — previously only
  cloudflare had them).
- 18 per-SDK `client.ts` / `client/api.ts` files: each imports its
  local `Retry` and passes it as `retry: Retry as any` into the
  `makeAPI` config.

AWS continues to use its own client and `Retry` tag, unchanged.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…etry policy

Standardizes how SDKs surface server-provided retry hints. Core defines an
opaque `retryAfter: Duration.Duration` field on every retryable error
(`TooManyRequests`, `Locked`, `InternalServerError`, `BadGateway`,
`ServiceUnavailable`, `GatewayTimeout`) and ships parsing primitives for the
two common wire formats:

  - `parseRetryAfter` — RFC 7231 §7.1.3: `delay-seconds` integer or HTTP-date
  - `parseRatelimit`  — IETF httpapi-ratelimit-headers: `r=0, t=30`
  - `parseServerRetryHint` — convenience: try one then the other

Each SDK's `matchError` now accepts an optional `headers` arg and is
responsible for deriving a Duration from whatever the service actually
returns (header, body field, bespoke shape, etc.). The hand-written matchers
in cloudflare/gcp/kubernetes and the 15 scaffolded SDKs (turso, planetscale,
neon, stripe, axiom, posthog, supabase, coinbase, fly-io, mongodb-atlas,
typesense, workos, prisma-postgres, azure, expo-eas) all populate
`retryAfter` via `parseServerRetryHint(headers)` for the standard cases. The
`scripts/create-sdk.ts` template emits the same wiring so newly scaffolded
SDKs honor server hints by default.

The default retry policy (`makeDefault`) and the new `throttlingFactory` /
`transientFactory` honor `error.retryAfter` with precedence over the
exponential backoff, capped at 60s so a misbehaving server can't park us
forever. AWS uses its own retry stack and is intentionally untouched.

Adds 29 unit tests covering header parsing edge cases (seconds, HTTP-date,
past/future, garbage, IETF Ratelimit) and the policy honoring `retryAfter`.

Pipeline awareness: prompts in `scripts/error-discovery.ts` and
`scripts/generate-tests.ts` instruct sub-agents to preserve the `headers`
parameter on `matchError` and pass `retryAfter` into retryable error
constructors when patching, so the convention sticks across regeneration.

Co-authored-by: Cursor <cursoragent@cursor.com>
@alchemy-version-bot

alchemy-version-bot Bot commented May 4, 2026

Copy link
Copy Markdown
Contributor

Install the packages built from this commit:

@distilled.cloud/core

bun add https://pkg.distilled.cloud/core/fc76631

@distilled.cloud/aws

bun add https://pkg.distilled.cloud/aws/fc76631

@distilled.cloud/cloudflare

bun add https://pkg.distilled.cloud/cloudflare/fc76631

@distilled.cloud/gcp

bun add https://pkg.distilled.cloud/gcp/fc76631

@distilled.cloud/neon

bun add https://pkg.distilled.cloud/neon/fc76631

@distilled.cloud/planetscale

bun add https://pkg.distilled.cloud/planetscale/fc76631

@distilled.cloud/prisma-postgres

bun add https://pkg.distilled.cloud/prisma-postgres/fc76631

@distilled.cloud/stripe

bun add https://pkg.distilled.cloud/stripe/fc76631

@distilled.cloud/supabase

bun add https://pkg.distilled.cloud/supabase/fc76631

@distilled.cloud/posthog

bun add https://pkg.distilled.cloud/posthog/fc76631

@distilled.cloud/axiom

bun add https://pkg.distilled.cloud/axiom/fc76631

@distilled.cloud/azure

bun add https://pkg.distilled.cloud/azure/fc76631

@distilled.cloud/kubernetes

bun add https://pkg.distilled.cloud/kubernetes/fc76631

@distilled.cloud/coinbase

bun add https://pkg.distilled.cloud/coinbase/fc76631

@distilled.cloud/mongodb-atlas

bun add https://pkg.distilled.cloud/mongodb-atlas/fc76631

@distilled.cloud/fly-io

bun add https://pkg.distilled.cloud/fly-io/fc76631

@distilled.cloud/turso

bun add https://pkg.distilled.cloud/turso/fc76631

@distilled.cloud/typesense

bun add https://pkg.distilled.cloud/typesense/fc76631

@distilled.cloud/workos

bun add https://pkg.distilled.cloud/workos/fc76631

@distilled.cloud/expo-eas

bun add https://pkg.distilled.cloud/expo-eas/fc76631

…yable errors

`HTTP_STATUS_MAP` includes both retryable (423/429/5xx) and non-retryable
(400/401/403/404/409/422) classes. The non-retryable schemas don't declare
`retryAfter` — Effect's `Schema.TaggedErrorClass` silently retains extra
constructor props on the instance, so passing `retryAfter` through the
generic `HTTP_STATUS_MAP[status]` dispatch was attaching a dead field that
would leak into `JSON.stringify` output on errors like `BadRequest`.

Adds `parseRetryAfterForStatus(status, headers)` in core that returns the
parsed Duration only when status is in `RETRYABLE_HTTP_STATUSES`
(`{423, 429, 500, 502, 503, 504}`). Switches every SDK matcher (and the
create-sdk template + agent-prompt scripts) to use the gated helper.

Cloudflare's `GLOBAL_ERROR_CODE_MAP[971]` (envelope rate-limit code returned
with arbitrary HTTP status, often 200) keeps using `parseServerRetryHint`
since `TooManyRequests` always declares `retryAfter`.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread packages/core/src/retry.ts
Comment thread scripts/error-discovery.ts
…ional retryAfter

- Replace hard-coded 60s server-hint cap with `DISTILLED_SERVER_RETRY_HINT_CAP_MS`
  (env) and optional `ServerRetryHintCapMs` Context service + `serverRetryHintCapLayer`.
- Document defaults via `DEFAULT_SERVER_RETRY_HINT_CAP_MS` and
  `readServerRetryHintCapMsFromEnv`.
- Soften error-discovery / generate-tests / create-sdk / create-sdk-full prompts:
  `retryAfter` is optional when headers lack a hint; default policy still
  exponential-backoffs without it.
- Expand `ClientConfig.matchError` JSDoc on cap tuning.
- Add `server-retry-hint-cap.test.ts` for env parsing and layer wiring.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Mkassabov Mkassabov added force-ci Bypass paths-filter and run CI for every package and removed force-ci Bypass paths-filter and run CI for every package labels May 4, 2026
@sam-goodwin
sam-goodwin changed the base branch from feat/cloudflare-blanket-retry to main May 4, 2026 03:15
@sam-goodwin
sam-goodwin merged commit 895697f into main May 4, 2026
40 of 45 checks passed
@sam-goodwin
sam-goodwin deleted the feat/retry-after-headers branch May 4, 2026 05:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants