feat(core): parse Retry-After / Ratelimit headers and honor them in retry policy#228
Merged
Conversation
The shared core `makeAPI` only ran a hard-coded throttling-only retry,
so Cloudflare's exported `Retry` Context.Service (`policy`, `none`,
`throttling`, `transient`) was effectively dead code. Callers had to
wrap individual operations with `Effect.retry({ ... })` instead of
installing one blanket policy at the layer level (the pattern AWS has
in `packages/aws/src/client/api.ts`).
Add an optional `retry?: Context.Key<any, RetryPolicy>` field to
`ClientConfig`. When set, every operation reads the policy via
`Effect.serviceOption(retry)` with `Retry.makeDefault` as the fallback
and a per-call `lastError` Ref, mirroring AWS's `outerFn`. When unset,
the legacy throttling-only schedule still runs, so other consumers
(turso, planetscale, neon, stripe, kubernetes, etc.) are unchanged
until they opt in.
Wire `retry: Retry` into Cloudflare's `makeAPI` so callers can do
`myEffect.pipe(Cloudflare.Retry.transient)` once and have every
Cloudflare API call below it transparently retry transient errors.
Co-authored-by: Cursor <cursoragent@cursor.com>
Per-SDK Retry tags were never actually consumed: each
`packages/<sdk>/src/retry.ts` declared its own `Retry extends
Context.Service<...>("FooRetry")`, but the shared `makeAPI` in
`@distilled.cloud/core/client` had no way to read from N different tags
and so just ran a hard-coded throttling-only schedule. The previous
commit on this branch threaded a per-SDK tag through `makeAPI` config —
that worked but it left every other SDK on the legacy throttling-only
path until they migrated.
Drop the per-SDK indirection entirely:
- `packages/core/src/retry.ts` now exports a single shared `Retry`
Context.Service plus `policy`, `none`, `throttling`, `transient`
helpers bound to it. `makeRetryService` is kept as a deprecated
no-op shim for any external callers.
- `packages/core/src/client.ts`'s `makeAPI` always reads the shared
`Retry` with `Effect.serviceOption`, falling back to
`Retry.makeDefault` (transient/throttling/server with capped
exponential backoff + jitter, 5 attempts). The `config.retry` field
introduced in the previous commit and the legacy hard-coded
throttling-only branch are removed. This mirrors AWS's pattern in
`packages/aws/src/client/api.ts`.
- All 17 per-SDK `retry.ts` files (cloudflare, turso, planetscale,
neon, stripe, kubernetes, axiom, posthog, gcp, coinbase, supabase,
expo-eas, fly-io, mongodb-atlas, typesense, workos, prisma-postgres,
azure) become thin re-exports of the shared core module so existing
call sites (`Cloudflare.Retry.transient(eff)`,
`Layer.succeed(Turso.Retry.Retry, customPolicy)`, etc.) keep working
but now actually take effect.
AWS continues to use its own client and its own `Retry` tag —
unchanged in this PR.
Co-authored-by: Cursor <cursoragent@cursor.com>
…client Reverts the shared-`Retry` approach from the previous commit. Each SDK keeps its own `Retry` Context.Service tag (so policies don't bleed across SDKs — `Cloudflare.Retry.transient` only retries Cloudflare calls, not turso underneath), but each SDK's `client.ts` now wires its own tag into `makeAPI` automatically. End users just write `myEffect.pipe(Cloudflare.Retry.transient)` — no per-SDK config to remember. - `packages/core/src/client.ts`: `ClientConfig.retry` is now required. `makeAPI` always reads the configured tag with `Effect.serviceOption` and falls back to `Retry.makeDefault`. Removed the legacy throttling-only branch. - `packages/core/src/retry.ts`: shared `Retry` tag/`policy`/`none`/ `throttling`/`transient` removed; `makeRetryService` restored as the primary API for SDKs. - 18 per-SDK `retry.ts` files: each declares its own `Retry` tag plus matching `policy`, `none`, `throttling`, `transient` helpers (added `throttling`/`transient` everywhere for parity — previously only cloudflare had them). - 18 per-SDK `client.ts` / `client/api.ts` files: each imports its local `Retry` and passes it as `retry: Retry as any` into the `makeAPI` config. AWS continues to use its own client and `Retry` tag, unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…etry policy Standardizes how SDKs surface server-provided retry hints. Core defines an opaque `retryAfter: Duration.Duration` field on every retryable error (`TooManyRequests`, `Locked`, `InternalServerError`, `BadGateway`, `ServiceUnavailable`, `GatewayTimeout`) and ships parsing primitives for the two common wire formats: - `parseRetryAfter` — RFC 7231 §7.1.3: `delay-seconds` integer or HTTP-date - `parseRatelimit` — IETF httpapi-ratelimit-headers: `r=0, t=30` - `parseServerRetryHint` — convenience: try one then the other Each SDK's `matchError` now accepts an optional `headers` arg and is responsible for deriving a Duration from whatever the service actually returns (header, body field, bespoke shape, etc.). The hand-written matchers in cloudflare/gcp/kubernetes and the 15 scaffolded SDKs (turso, planetscale, neon, stripe, axiom, posthog, supabase, coinbase, fly-io, mongodb-atlas, typesense, workos, prisma-postgres, azure, expo-eas) all populate `retryAfter` via `parseServerRetryHint(headers)` for the standard cases. The `scripts/create-sdk.ts` template emits the same wiring so newly scaffolded SDKs honor server hints by default. The default retry policy (`makeDefault`) and the new `throttlingFactory` / `transientFactory` honor `error.retryAfter` with precedence over the exponential backoff, capped at 60s so a misbehaving server can't park us forever. AWS uses its own retry stack and is intentionally untouched. Adds 29 unit tests covering header parsing edge cases (seconds, HTTP-date, past/future, garbage, IETF Ratelimit) and the policy honoring `retryAfter`. Pipeline awareness: prompts in `scripts/error-discovery.ts` and `scripts/generate-tests.ts` instruct sub-agents to preserve the `headers` parameter on `matchError` and pass `retryAfter` into retryable error constructors when patching, so the convention sticks across regeneration. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
|
Install the packages built from this commit: @distilled.cloud/core bun add https://pkg.distilled.cloud/core/fc76631@distilled.cloud/aws bun add https://pkg.distilled.cloud/aws/fc76631@distilled.cloud/cloudflare bun add https://pkg.distilled.cloud/cloudflare/fc76631@distilled.cloud/gcp bun add https://pkg.distilled.cloud/gcp/fc76631@distilled.cloud/neon bun add https://pkg.distilled.cloud/neon/fc76631@distilled.cloud/planetscale bun add https://pkg.distilled.cloud/planetscale/fc76631@distilled.cloud/prisma-postgres bun add https://pkg.distilled.cloud/prisma-postgres/fc76631@distilled.cloud/stripe bun add https://pkg.distilled.cloud/stripe/fc76631@distilled.cloud/supabase bun add https://pkg.distilled.cloud/supabase/fc76631@distilled.cloud/posthog bun add https://pkg.distilled.cloud/posthog/fc76631@distilled.cloud/axiom bun add https://pkg.distilled.cloud/axiom/fc76631@distilled.cloud/azure bun add https://pkg.distilled.cloud/azure/fc76631@distilled.cloud/kubernetes bun add https://pkg.distilled.cloud/kubernetes/fc76631@distilled.cloud/coinbase bun add https://pkg.distilled.cloud/coinbase/fc76631@distilled.cloud/mongodb-atlas bun add https://pkg.distilled.cloud/mongodb-atlas/fc76631@distilled.cloud/fly-io bun add https://pkg.distilled.cloud/fly-io/fc76631@distilled.cloud/turso bun add https://pkg.distilled.cloud/turso/fc76631@distilled.cloud/typesense bun add https://pkg.distilled.cloud/typesense/fc76631@distilled.cloud/workos bun add https://pkg.distilled.cloud/workos/fc76631@distilled.cloud/expo-eas bun add https://pkg.distilled.cloud/expo-eas/fc76631 |
…yable errors
`HTTP_STATUS_MAP` includes both retryable (423/429/5xx) and non-retryable
(400/401/403/404/409/422) classes. The non-retryable schemas don't declare
`retryAfter` — Effect's `Schema.TaggedErrorClass` silently retains extra
constructor props on the instance, so passing `retryAfter` through the
generic `HTTP_STATUS_MAP[status]` dispatch was attaching a dead field that
would leak into `JSON.stringify` output on errors like `BadRequest`.
Adds `parseRetryAfterForStatus(status, headers)` in core that returns the
parsed Duration only when status is in `RETRYABLE_HTTP_STATUSES`
(`{423, 429, 500, 502, 503, 504}`). Switches every SDK matcher (and the
create-sdk template + agent-prompt scripts) to use the gated helper.
Cloudflare's `GLOBAL_ERROR_CODE_MAP[971]` (envelope rate-limit code returned
with arbitrary HTTP status, often 200) keeps using `parseServerRetryHint`
since `TooManyRequests` always declares `retryAfter`.
Co-authored-by: Cursor <cursoragent@cursor.com>
Mkassabov
reviewed
May 4, 2026
Mkassabov
reviewed
May 4, 2026
…ional retryAfter - Replace hard-coded 60s server-hint cap with `DISTILLED_SERVER_RETRY_HINT_CAP_MS` (env) and optional `ServerRetryHintCapMs` Context service + `serverRetryHintCapLayer`. - Document defaults via `DEFAULT_SERVER_RETRY_HINT_CAP_MS` and `readServerRetryHintCapMsFromEnv`. - Soften error-discovery / generate-tests / create-sdk / create-sdk-full prompts: `retryAfter` is optional when headers lack a hint; default policy still exponential-backoffs without it. - Expand `ClientConfig.matchError` JSDoc on cap tuning. - Add `server-retry-hint-cap.test.ts` for env parsing and layer wiring. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Standardizes how SDKs surface server-provided retry hints. Stacks on top of #225.
retryAfter: Duration.Durationfield on every retryable error (TooManyRequests,Locked,InternalServerError,BadGateway,ServiceUnavailable,GatewayTimeout).parseRetryAfter(RFC 7231 §7.1.3 —delay-secondsor HTTP-date),parseRatelimit(IETF httpapi-ratelimit-headers —r=0, t=30), andparseServerRetryHint(try one then the other).matchErrornow accepts an optionalheadersarg and is responsible for deriving aDurationfrom whatever the service actually returns. Cloudflare, GCP, Kubernetes, plus all 15 scaffolded SDKs (turso, planetscale, neon, stripe, axiom, posthog, supabase, coinbase, fly-io, mongodb-atlas, typesense, workos, prisma-postgres, azure, expo-eas) populateretryAfterviaparseServerRetryHint(headers)for the standard cases. Thescripts/create-sdk.tstemplate emits the same wiring so newly scaffolded SDKs honor server hints by default.makeDefault) and the newthrottlingFactory/transientFactoryhonorerror.retryAfterwith precedence over the exponential backoff, capped at 60s so a misbehaving server can't park us forever.scripts/error-discovery.tsandscripts/generate-tests.tsteach sub-agents to preserve theheadersparameter onmatchErrorand passretryAfterinto retryable error constructors when patching, so the convention sticks across regeneration.Override seam
If a service uses bespoke headers or body fields for cooldown, edit the SDK's
matchError:Test plan
bun run testinpackages/core— 29 new unit tests pass (header parsing edge cases + policy honoringretryAfter)bunx tsgo --noEmitclean for cloudflare, kubernetes, turso, planetscale, neon, stripe, axiom, posthog, supabase, coinbase, fly-io, mongodb-atlas, typesense, workos, prisma-postgres, azure, expo-easbunx oxlint srcclean across all 18 SDKsmain, out of scope)Made with Cursor