Skip to content

fix(compaction): keep the failure record a projection is re-read through - #8432

Merged
esengine merged 1 commit into
main-v2from
fix/failure-record-survives-repeated-folds
Aug 12, 2026
Merged

fix(compaction): keep the failure record a projection is re-read through#8432
esengine merged 1 commit into
main-v2from
fix/failure-record-survives-repeated-folds

Conversation

@esengine

Copy link
Copy Markdown
Owner

Found while auditing a suspicion raised during the #8419 / #8424 work. The
suspicion (that KeepErrors content accumulates without bound) turned out to be
wrong — failure_snip.go already caps each failure at 60 lines and its own
comment acknowledges the retention is deliberate. The real defect was the
opposite: the protection expires.

Measured behaviour, before

Fixture shaped like real go test output, KeepErrors on, three folds:

Fold Failure in projection ToolExecution
1 yes intact
2 yes stripped
3 gone

Why

KeepErrors classifies a failure from ToolExecution, not from message text —
that change was made deliberately because text matching missed real failures. A
failing go test log opens with === RUN, so the error: / blocked: prefix
arm cannot see it.

checkpointProjectionMessages wrote the projection through ModelMessages,
which strips ToolExecution:

// Local shell metadata must never enter provider request bytes.
candidate.ToolExecution = nil

That strip is correct for a provider request. The problem is that a
projection is read twice: once as what the model is sent, and once as the input
the next compaction classifies. Only the second reading needs the host's record.
Writing the projection through the provider-boundary function collapsed those two
roles, so fold 2 saw a tool result with no failure record and text matching
nothing, folded it into the digest, and the model stopped being told.

Note this makes the write-time strip not merely harmful but redundant:
context_manager.go:190 already builds every request through ModelMessages on
its own.

Change

ProjectionMessages — the same projection, ToolExecution preserved. The strip
stays at the provider boundary where it belongs. Both functions and their shared
implementation move to provider/projection.go: deriving the two copies of a
transcript is one responsibility, and provider.go was already carrying recorded
file-size debt that keeping them there would have widened.

Tests

The regression test uses a === RUN fixture on purpose. A synthetic
"error: ..." body survives either way through the text-prefix arm — which is
exactly what let this sit unnoticed. It asserts three things per fold: the
failure is still in the projection, its record is still readable, and no
ToolExecution reaches the provider request.

Verified by mutation, not just by passing: restoring ModelMessages at the write
site turns it red at fold 2 with the failure record was stripped, so the next fold cannot classify it. Restored and re-verified green.

Compatibility

ToolExecution is already persisted in canonical transcripts (its own doc
comment says so), so no new field enters the sidecar format and no privacy
boundary moves. A projection written by an older build simply has no record to
read, degrading to today's behaviour rather than failing.

Verification

gofmt -l .        clean
golangci-lint     0 issues
repolint          113538 — 33 lines BELOW the main-v2 baseline (113571)
go test ./...     all green, zero failures

Cache-impact: none - the provider-visible bytes are byte-identical. ProjectionMessages differs from ModelMessages only in a field that ModelMessages strips again at every request boundary, so no request payload, prompt-cache prefix, or projection fingerprint changes; providerVisibleFingerprint calls ModelMessages itself.
Cache-guard: internal/agent/failure_survives_folds_test.go asserts no ToolExecution reaches provider.ModelMessages output on every fold, alongside the unchanged internal/agent/cachehit_e2e_test.go.
Documentation-impact: updated - docs/SPEC.md "What survives a fold" now explains why a stored projection keeps the ToolExecution record that a provider request does not, with the matching section in docs/SPEC.zh-CN.md.

KeepErrors classifies a failure from ToolExecution rather than from message
text, because text matching was proven to miss real failures: a failing
`go test` log opens with "=== RUN", and no prefix match can see it. But
checkpointProjectionMessages wrote the projection through ModelMessages, which
strips ToolExecution on the way to a provider — so the record survived exactly
one fold. The second fold saw a tool result with no failure record and text
that matches nothing, folded it into the digest, and the model stopped being
told about the failure.

Measured on a fixture shaped like real `go test` output: present after fold 1,
record stripped after fold 2, gone from the projection after fold 3.

A projection is read twice — as what the model is sent, and as the input the
next compaction classifies — and only the second reading needs the host's own
record. So the strip belongs at the provider boundary, which every request path
already crosses (context_manager.go builds each request through ModelMessages
of its own accord, making the strip at write time redundant as well as harmful).
ProjectionMessages is that same projection with ToolExecution preserved.

Both functions and their shared implementation move to provider/projection.go:
deriving the two copies of a transcript is one responsibility, and provider.go
was already carrying recorded file-size debt this would have widened.

The regression test uses a "=== RUN" fixture on purpose. A synthetic
"error: ..." body survives either way through the text-prefix arm, which is
what let this go unnoticed. Verified by mutation: restoring ModelMessages at
the write site turns it red at fold 2.

Compatibility holds in both directions: ToolExecution is already persisted in
canonical transcripts, so no new field enters the sidecar format, and a
projection written by an older build simply has no record to read.
@esengine
esengine requested a review from SivanCola as a code owner August 12, 2026 00:13
@github-actions github-actions Bot added agent Core agent loop (internal/agent, internal/control) provider Model providers & selection (internal/provider) v2 Go rewrite (1.x) — main-v2 branch, active development labels Aug 12, 2026
@esengine
esengine merged commit ff7b276 into main-v2 Aug 12, 2026
24 checks passed
@esengine
esengine deleted the fix/failure-record-survives-repeated-folds branch August 12, 2026 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Core agent loop (internal/agent, internal/control) provider Model providers & selection (internal/provider) v2 Go rewrite (1.x) — main-v2 branch, active development

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant