agent: an unregistered mitigation name is dropped before validation, so a name-resolution failure is recorded as a genuine agent_stop

[vivekkhandelwal1]

Summary

When the LLM proposer returns a mitigation name that is not in the registry, that name is silently dropped before any validation runs. If it was the only name proposed, next_mitigations becomes empty, and the loop's stop check reads an empty list as "the agent has no further hypotheses". The search ends with outcome = agent_stop and stop_reason = agent_requested, and the operator is shown the model's own hypothesis as the reason the search stopped.

Nothing raises, nothing is logged as a rejection, and the exit status is success. A model that emitted a slightly wrong name is indistinguishable, in every recorded field, from a model that correctly concluded it had run out of ideas.

The sharpest form of the problem: in the reproduction below the model explicitly sets "stop": false, and the loop still records stop_reason = agent_requested. The recorded reason directly contradicts the model's own output.

Verified against main at 90dae93. All line numbers are from that commit.

The precise code path

The drop happens inside the proposer, not the loop:

1. LiteLLMProposer.propose() calls the model and parses the reply with AgentStep.from_dict(). At this point next_mitigations still holds whatever the model said.

2. llm.py:273-274 filters against the remaining candidate set and returns the filtered list:

   # Filter to remaining candidates only
   filtered = [m for m in step.next_mitigations if m in remaining]

   An unregistered name is not in remaining, so it is discarded here. The rejected names are not retained anywhere on the returned AgentStep.

3. Back in the loop, loop.py:433 calls AgentPolicy.validate_step(). That is the function that would have caught the bad name: policy.py:85-91 calls get_mitigation(name) and converts UnknownMitigationError into a PolicyViolation. But it iterates step.next_mitigations, which is now empty, so it validates nothing and returns cleanly.

4. loop.py:449 is if step.stop or not step.next_mitigations: — the empty list alone satisfies the condition, regardless of step.stop.

5. _resolve_stop_outcome() derives the reason. step.stop_reason is None, the baseline did not pass, and the exhausted_candidates branch requires "No remaining" in step.hypothesis — which a real model's prose will not contain. So it falls through to agent_requested, and the search_stopped log event records that reason.

The ordering is the whole defect: the filter at step 2 runs before the validation at step 3, and empties the input that validation would have rejected.

It also defeats an existing guard

loop.py:460-472 already contains a guard for the adjacent case, with a comment stating the intent plainly:

a custom/buggy proposer must not run a registered-but-not-allowlisted mitigation. Fail fast [...] instead of silently widening the search.

That guard sits after the stop check at line 449, so it is unreachable for any name the proposer already filtered out. The loop is written to fail fast on a misbehaving proposer, and _resolve_stop_outcome even downgrades a falsely-claimed baseline_pass for the same reason — but this path bypasses both.

Note also that the filter exists only in LiteLLMProposer; FakeLLMProposer selects from candidates itself and cannot produce an unregistered name. So the filter is present exactly on the path where a name can actually be hallucinated, and LiteLLMProposer currently has no test coverage (git grep LiteLLMProposer tests/ returns nothing), which is presumably why it has gone unnoticed.

Minimal reproduction

No GPU, no container, no network, and no litellm install. Only the HTTP transport is stubbed, so llm.py, policy.py and loop.py are all real code. Save as repro_issue_snippet.py in the repo root and run PYTHONPATH=src python3 repro_issue_snippet.py.

import sys
import types
from unittest import mock

from aorta.agent.llm import LiteLLMProposer
from aorta.agent.loop import _resolve_stop_outcome
from aorta.agent.policy import AgentPolicy

SUMMARIES = [{"cell_name": "none-none", "verdict": "fail",
              "failure_detectors_fired": ["tier2:hang"],
              "warn_detectors_fired": [], "capture": {}, "exit_code": None}]
CANDIDATES = ["nccl_launch_order_implicit", "hsa_no_sdma", "tf32_off"]


def propose(content):
    """Call the real LiteLLMProposer with a canned model response."""
    litellm = types.ModuleType("litellm")
    litellm.completion = lambda **_: mock.Mock(
        choices=[mock.Mock(message=mock.Mock(content=content))])
    with mock.patch.dict(sys.modules, {"litellm": litellm}):
        return LiteLLMProposer(model="any").propose(
            symptom="training hangs at step 40",
            cell_summaries=SUMMARIES, candidates=CANDIDATES, tried=[])


def what_the_loop_does(label, content):
    step = AgentPolicy().validate_step(propose(content))  # loop.py:433
    print(f"--- {label}")
    print(f"    model asked for : {content}")
    print(f"    next_mitigations: {step.next_mitigations}    stop: {step.stop}")
    if step.stop or not step.next_mitigations:                # loop.py:449
        outcome, message, reason = _resolve_stop_outcome(step, SUMMARIES)
        print(f"    outcome         : {outcome}")
        print(f"    stop_reason     : {reason}")
        print(f"    operator is told: {message!r}")
    else:
        print(f"    loop continues  : {step.next_mitigations}")


# 'rccl_p2p_disable' is NOT one of the 22 registered mitigations. It is the
# kind of name a model invents because it reads exactly like one that exists.
what_the_loop_does("B: one unregistered name", (
    '{"category": "rccl_hang", "hypothesis": "peer-to-peer transport is '
    'wedged; disable it", "next_mitigations": ["rccl_p2p_disable"], '
    '"confidence": 0.9, "stop": false}'))

what_the_loop_does("D: a genuine stop, for comparison", (
    '{"category": "unknown", "hypothesis": "no further hypotheses from this '
    'evidence", "next_mitigations": [], "confidence": 0.3, "stop": true}'))

Observed output:

--- B: one unregistered name
    model asked for : {"category": "rccl_hang", "hypothesis": "peer-to-peer transport is wedged; disable it", "next_mitigations": ["rccl_p2p_disable"], "confidence": 0.9, "stop": false}
    next_mitigations: []    stop: False
    outcome         : agent_stop
    stop_reason     : agent_requested
    operator is told: 'peer-to-peer transport is wedged; disable it'
--- D: a genuine stop, for comparison
    model asked for : {"category": "unknown", "hypothesis": "no further hypotheses from this evidence", "next_mitigations": [], "confidence": 0.3, "stop": true}
    next_mitigations: []    stop: True
    outcome         : agent_stop
    stop_reason     : agent_requested
    operator is told: 'no further hypotheses from this evidence'

B and D agree on outcome, on stop_reason, and on the search_stopped log event. B is an agent-side name-resolution failure; D is the model correctly reporting that it has no further hypotheses. Nothing downstream can tell them apart.

There is a quieter variant. If the model proposes one unregistered and one valid name, the valid one survives and the search continues, so the loop executes half of the plan the model actually stated, and no record of the discarded half is kept:

model returned  : {"next_mitigations": ["rccl_p2p_disable", "nccl_launch_order_implicit"], ...}
next_mitigations: ['nccl_launch_order_implicit']
loop continues  : ['nccl_launch_order_implicit']

Why it matters

In production, it misattributes an agent-side data problem to the model's reasoning. The operator reads "the agent stopped and here is its hypothesis" and concludes the model gave up on this evidence. The truth is that the loop could not resolve a name the model asked for. This sends debugging effort at the prompt or the model when the actual fault is in name resolution — and the hypothesis shown alongside the stop actively reinforces the wrong conclusion, because it is a plausible-sounding rationale for stopping. The failure mode this most resembles is the familiar class of bug where a success code hides an unperformed operation: the caller is told the operation completed, so it never checks whether anything happened. The diagnostic value of the agent loop rests on its stop reasons being trustworthy; if agent_stop can mean either "the model was done" or "the agent dropped the model's request", then no stop reason can be taken at face value, including the correct ones.

It silently corrupts any metric or reward computed from stop reasons. A model whose mitigation names are subtly wrong scores identically to a model that reasoned correctly to a genuine stop, so the signal that should penalise bad names is not just absent but inverted: emitting an unregistered name produces the same recorded outcome as a well-formed conclusion, and the wrong behaviour is scored as acceptable. Any aggregate over agent_stop — success rates, model comparisons, regression tracking across nightly runs — mixes the two populations with no way to separate them after the fact, since the rejected name is never written down.

Suggested remedies

Not prescribing one; the choice depends on how much proposer misbehaviour you want the loop to tolerate.

1. Raise on an unregistered name. Drop the filter at llm.py:274 and let validate_step() do its job. PolicyViolation is already caught by the loop and surfaces as policy_stop, so the machinery exists and the diagnostic is immediate. Consistent with the existing allowlist guard at loop.py:466, which chose exactly this for the adjacent case.
2. Keep filtering, but record it. Retain the rejected names on the AgentStep (e.g. a rejected_mitigations field), log a warning, and include them in the search_stopped event. Preserves the current tolerance for a sloppy model while making the drop auditable — and it also covers the mixed-list variant above, which remedy 1 leaves untouched.
3. Introduce a distinct terminal reason, such as invalid_mitigation, so this stop is separable from a genuine agent_stop in artifacts and in any aggregate computed over them.

I would pick 1, with the rejected_mitigations field from 2 alongside it. Removing the filter restores the guarantee the code already tries to make in two other places — a misbehaving proposer fails fast rather than silently narrowing or widening the search — and it needs no new vocabulary in the stop-reason enum, so nothing downstream has to learn a new value. Keeping the rejected names is what makes the mixed-list case visible too, and it is the piece that turns "the search stopped" into "the search stopped because this name did not resolve". Remedy 3 is a reasonable alternative if you would rather a bad name end the search cleanly than raise, but it is strictly more surface area than 1 for the same information.

Discovery context

Found while scaffolding reward functions for RL post-training against the agent's proposer contract. That work grades a model on whether its proposal is well-formed — parseable, correct schema, valid category, registered mitigation name — which meant reading the stop path closely enough to know what the loop actually does with a name it cannot resolve. The tier that should score "registered name" is precisely the tier the filter makes invisible, which is how the ordering surfaced. Reported as a defect rather than a fix PR because the right remedy is a maintainer's call about proposer tolerance, not mine.