Skip to content

Commit e6e06f5

Browse files
nicoloboschinicoloboschi
committed
fix(coding-agents): cap the 429 backoff so a long Retry-After cannot park a drain
Review follow-up on the retry path. `Retry-After` was honoured without a ceiling, so an hour-long value — an incident, a misconfigured limiter, a proxy inventing one — would park the drain for that hour, up to the whole `maxMs`, with the background seed frozen behind it. The header is a server's hint, not a budget we owe it. Capped at 60s. That keeps the signal (the floor and the header still lengthen the wait) without handing over the schedule: if the limit still applies, the next poll gets another 429 and backs off again. Also re-syncs the docs page and skill mirror from the README, which the rebase onto main left stale by a column width.
1 parent 22d8ee8 commit e6e06f5

5 files changed

Lines changed: 53 additions & 22 deletions

File tree

‎hindsight-docs/docs-integrations/coding-agents.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -281,7 +281,8 @@ Environment variables are a **fallback**: the file wins wherever it sets a value
281281
an existing setup changes nothing. `retainTags` takes a comma-separated list
282282
(`HINDSIGHT_RETAIN_TAGS="project:{gitProject},env:work"`); entries are trimmed and blanks dropped.
283283
The map-valued settings (`mapPathToBank`, `harnesses`, `banks`, `retainMetadata`) are file-only —
284-
per-key branching doesn't survive flattening into one variable.
284+
per-key branching doesn't survive flattening into one variable. `maxParallelRetains` is available
285+
as `HINDSIGHT_MAX_PARALLEL_RETAINS` for containers and CI.
285286

286287
There is deliberately no repo-carried config file — per-repo bank routing is `mapPathToBank`,
287288
per-agent differences are `harnesses.<name>`.
@@ -321,6 +322,7 @@ hook by Codex...), so one shared config serves several agents side by side:
321322
| `surveyModel` | `haiku` | model for the survey — Claude recipe only (`claude -p --model`); other agents use their configured default |
322323
| `surveyBudgetUsd` | `2` | survey spend cap — Claude recipe only (`claude -p --max-budget-usd`); other agents rely on their read-only sandbox |
323324
| `retainSessions` | `true` | plugin-harness write-back (opencode, Kilo): async upsert of the session transcript every turn, plus an idle flush that captures the reply the per-turn pass can't see (set `false` to opt out; hook harnesses always write on Stop) |
325+
| `maxParallelRetains` | `10` | cap on concurrent retain-related requests: drain()'s per-op polls plus deepen's chat/git retain pools. The API rate-limits bursts, not single requests — if you see 429s, lower this rather than raising it |
324326
| `logLevel` | `"info"` | plugin-log verbosity (`"debug"` \| `"info"` \| `"warn"` \| `"error"`); `HINDSIGHT_LOG_LEVEL` env overrides |
325327
| `gitIngest` | `"message"` | git depth for seeding AND staying current (same engine): `"message"` = commit messages only (one doc, re-upserted when HEAD moves); `"full"` = messages + per-commit full diffs (progressive, newest first); `"none"` = git off |
326328
| `harnesses.<name>` | — | per-harness override of any field above |

‎hindsight-integrations/coding-agents/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -315,7 +315,7 @@ hook by Codex...), so one shared config serves several agents side by side:
315315
| `surveyModel` | `haiku` | model for the survey — Claude recipe only (`claude -p --model`); other agents use their configured default |
316316
| `surveyBudgetUsd` | `2` | survey spend cap — Claude recipe only (`claude -p --max-budget-usd`); other agents rely on their read-only sandbox |
317317
| `retainSessions` | `true` | plugin-harness write-back (opencode, Kilo): async upsert of the session transcript every turn, plus an idle flush that captures the reply the per-turn pass can't see (set `false` to opt out; hook harnesses always write on Stop) |
318-
| `maxParallelRetains` | `10` | cap on concurrent retain-related requests: drain()'s per-op polls plus deepen's chat/git retain pools. The API rate-limits bursts, not single requests — if you see 429s, lower this rather than raising it |
318+
| `maxParallelRetains` | `10` | cap on concurrent retain-related requests: drain()'s per-op polls plus deepen's chat/git retain pools. The API rate-limits bursts, not single requests — if you see 429s, lower this rather than raising it |
319319
| `logLevel` | `"info"` | plugin-log verbosity (`"debug"` \| `"info"` \| `"warn"` \| `"error"`); `HINDSIGHT_LOG_LEVEL` env overrides |
320320
| `gitIngest` | `"message"` | git depth for seeding AND staying current (same engine): `"message"` = commit messages only (one doc, re-upserted when HEAD moves); `"full"` = messages + per-commit full diffs (progressive, newest first); `"none"` = git off |
321321
| `harnesses.<name>` | — | per-harness override of any field above |

‎hindsight-integrations/coding-agents/src/core/hindsight.test.ts‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,23 @@ describe("HindsightClient.drain", () => {
6464
await p;
6565
});
6666

67+
it("caps the backoff however long Retry-After asks for", async () => {
68+
// The header is a hint, not a budget we owe the server: an hour-long value would otherwise
69+
// park the drain — and the background seed behind it — for that hour.
70+
vi.useFakeTimers();
71+
const client = new HindsightClient({ apiUrl: "http://x", bank: "b" });
72+
const fetchMock = vi.fn(async () => jsonResponse(429, {}, { "Retry-After": "3600" }));
73+
vi.stubGlobal("fetch", fetchMock);
74+
75+
const p = client.drain(["1"], "test", 300_000);
76+
await vi.advanceTimersByTimeAsync(0);
77+
expect(fetchMock).toHaveBeenCalledTimes(1);
78+
await vi.advanceTimersByTimeAsync(60_000); // capped at 60s, not 3600s
79+
expect(fetchMock).toHaveBeenCalledTimes(2);
80+
await vi.advanceTimersByTimeAsync(300_000);
81+
await p;
82+
});
83+
6784
it("uses the 10s floor when Retry-After is shorter than it", async () => {
6885
vi.useFakeTimers();
6986
const client = new HindsightClient({ apiUrl: "http://x", bank: "b" });

‎hindsight-integrations/coding-agents/src/core/hindsight.ts‎

Lines changed: 29 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,17 @@ const POLL_CYCLE_MS = 5000;
6969
/** Minimum backoff after a 429 that carried no (or a shorter) Retry-After. */
7070
const RETRY_AFTER_FLOOR_MS = 10 * 1000;
7171

72+
/**
73+
* Ceiling on a single backoff, however long `Retry-After` asks for.
74+
*
75+
* The header is a server's hint, not a budget we owe it: a large value (an incident, a
76+
* misconfigured limiter, a proxy inventing one) would otherwise park a drain for as long as it
77+
* says — up to the whole `maxMs`, with the background seed frozen behind it. Capping keeps the
78+
* signal without handing over the schedule; if the limit still applies, the next poll simply gets
79+
* another 429 and backs off again.
80+
*/
81+
const RETRY_AFTER_CEILING_MS = 60 * 1000;
82+
7283
/** Parse a `Retry-After` header (delta-seconds or HTTP-date) into milliseconds; 0 when absent. */
7384
export function retryAfterMs(header: string | null | undefined): number {
7485
if (!header) return 0;
@@ -301,27 +312,26 @@ export class HindsightClient {
301312
// Cycle backoff: default 5s; any 429 in the cycle raises it to the longest Retry-After seen
302313
// (floor 10s) so a rate-limited API gets room to recover before the next poll round.
303314
let backoffMs = POLL_CYCLE_MS;
304-
await pool(
305-
[...pending],
306-
this.maxParallelRetains,
307-
async (id) => {
308-
try {
309-
const r = await fetch(this.bankUrl(`/operations/${id}`), { headers: this.headers() });
310-
if (r.status === 429) {
311-
backoffMs = Math.max(backoffMs, RETRY_AFTER_FLOOR_MS, retryAfterMs(r.headers.get("retry-after")));
312-
return; // op stays pending — retried after the backoff
313-
}
314-
if (!r.ok) return;
315-
const st = (((await r.json()) as { status?: string }).status || "").toLowerCase();
316-
if (TERMINAL.has(st)) {
317-
pending.delete(id);
318-
if (st !== "completed") failed++;
319-
}
320-
} catch {
321-
/* transient — retry next cycle */
315+
await pool([...pending], this.maxParallelRetains, async (id) => {
316+
try {
317+
const r = await fetch(this.bankUrl(`/operations/${id}`), { headers: this.headers() });
318+
if (r.status === 429) {
319+
backoffMs = Math.min(
320+
RETRY_AFTER_CEILING_MS,
321+
Math.max(backoffMs, RETRY_AFTER_FLOOR_MS, retryAfterMs(r.headers.get("retry-after")))
322+
);
323+
return; // op stays pending — retried after the backoff
322324
}
325+
if (!r.ok) return;
326+
const st = (((await r.json()) as { status?: string }).status || "").toLowerCase();
327+
if (TERMINAL.has(st)) {
328+
pending.delete(id);
329+
if (st !== "completed") failed++;
330+
}
331+
} catch {
332+
/* transient — retry next cycle */
323333
}
324-
);
334+
});
325335
if (pending.size) {
326336
this.log(` … ${pending.size}/${ids.length} ${label} ops pending`);
327337
await sleep(backoffMs);

‎skills/hindsight-docs/references/sdks/integrations/coding-agents.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -276,7 +276,8 @@ Environment variables are a **fallback**: the file wins wherever it sets a value
276276
an existing setup changes nothing. `retainTags` takes a comma-separated list
277277
(`HINDSIGHT_RETAIN_TAGS="project:{gitProject},env:work"`); entries are trimmed and blanks dropped.
278278
The map-valued settings (`mapPathToBank`, `harnesses`, `banks`, `retainMetadata`) are file-only —
279-
per-key branching doesn't survive flattening into one variable.
279+
per-key branching doesn't survive flattening into one variable. `maxParallelRetains` is available
280+
as `HINDSIGHT_MAX_PARALLEL_RETAINS` for containers and CI.
280281

281282
There is deliberately no repo-carried config file — per-repo bank routing is `mapPathToBank`,
282283
per-agent differences are `harnesses.<name>`.
@@ -316,6 +317,7 @@ hook by Codex...), so one shared config serves several agents side by side:
316317
| `surveyModel` | `haiku` | model for the survey — Claude recipe only (`claude -p --model`); other agents use their configured default |
317318
| `surveyBudgetUsd` | `2` | survey spend cap — Claude recipe only (`claude -p --max-budget-usd`); other agents rely on their read-only sandbox |
318319
| `retainSessions` | `true` | plugin-harness write-back (opencode, Kilo): async upsert of the session transcript every turn, plus an idle flush that captures the reply the per-turn pass can't see (set `false` to opt out; hook harnesses always write on Stop) |
320+
| `maxParallelRetains` | `10` | cap on concurrent retain-related requests: drain()'s per-op polls plus deepen's chat/git retain pools. The API rate-limits bursts, not single requests — if you see 429s, lower this rather than raising it |
319321
| `logLevel` | `"info"` | plugin-log verbosity (`"debug"` \| `"info"` \| `"warn"` \| `"error"`); `HINDSIGHT_LOG_LEVEL` env overrides |
320322
| `gitIngest` | `"message"` | git depth for seeding AND staying current (same engine): `"message"` = commit messages only (one doc, re-upserted when HEAD moves); `"full"` = messages + per-commit full diffs (progressive, newest first); `"none"` = git off |
321323
| `harnesses.<name>` | — | per-harness override of any field above |

0 commit comments

Comments
 (0)