Opt-in fail-closed integrity gate for repaired or truncated derivations

[WaterR47]

Summary

A length-truncated structured-output response can currently be repaired into a schema-valid PromptRepresentation and then persisted as memory even though the model did not complete the derivation. For persistent conclusions, this turns an availability-oriented repair into a memory-integrity risk.

I propose an opt-in deriver setting that accepts only complete responses at the persistence boundary, rejects the repair marker, retries the derivation a bounded number of times, and then surfaces an error without saving anything.

Current path

At commit 4f9a41360a853e520ddc6a8aecf851ca04137da3:

1. src/llm/backends/openai.py catches LengthFinishReasonError on structured output and repairs the partial response rather than failing the call.
2. src/llm/structured_output.py (repair_response_model_json) can fill a missing deductive conclusion with text beginning [Incomplete reasoning.
3. src/deriver/deriver.py calls honcho_llm_call, converts response.content, and enters the observer save loop without checking HonchoLLMCallResponse.finish_reasons (src/llm/types.py).
4. honcho_llm_call(..., enable_retry=True) retries exceptions, but successful repair of a truncated response does not raise, so that retry mechanism does not cover this case.

The repair behavior may be appropriate for availability-sensitive conversational flows. The concern is specifically using repaired output as durable memory.

Proposed behavior

Add an opt-in [deriver] FAIL_CLOSED setting, defaulting to false for backward compatibility. When enabled at the deriver save seam:

• consider an empty finish-reason list or any reason other than stop incomplete;
• reject any generated observation text containing [Incomplete reasoning, even if the reported finish reason is stop;
• make at most two fresh derivation retries after the initial response;
• after the bounded attempts, emit an ERROR with workspace/session/message identifiers and finish state, increment a labeled Prometheus failure counter when metrics are enabled, and raise before conversion or persistence begins.

This keeps the backend repair path unchanged for other consumers while allowing memory operators to prefer integrity over availability.

Why the save seam

The response finish state and repaired PromptRepresentation are both available immediately after honcho_llm_call. Rejecting there guarantees that no observer collection save has started. It also keeps the behavior narrowly scoped to persistent derivations rather than changing dialectic or generic structured-output behavior.

Suggested tests

• empty and non-stop finish reasons are retried, then raise without calling save_representation;
• a complete stop response follows the existing save path unchanged;
• a response containing [Incomplete reasoning is rejected before persistence even with finish_reasons == ["stop"];
• a later complete retry is persisted exactly once;
• disabled/default setting preserves current behavior.

I have a local patch against 4f9a4136 implementing this shape and can prepare it as a PR if maintainers are receptive.