Conversation
Collaborator
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: feiiiiii5 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
The verifier builds a chat request (messages= plus choices[0].message.content) but sends it to client.completions.create, which takes prompt instead. The resulting error is swallowed by the bare except, so semantic verification always returned None and every hit fell through to a real LLM call. Route the request to client.chat.completions (openai>=1.0) or ChatCompletion (legacy module client), which is what the existing unit test and the example expect. Signed-off-by: feiiiiii5 <204683769+feiiiiii5@users.noreply.github.com>
feiiiiii5
force-pushed
the
fix/llm-verifier-chat-endpoint
branch
from
September 20, 2026 08:07
0c18da1 to
e37f0e7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
LlmVerifier/llm_semantic_verificationnever returns a verified answer. The function builds achat request — it sends
messages=and readsresp.choices[0].message.content— but then sendsit to the completions endpoint:
completions.createtakesprompt, notmessages, so the call raises; the bareexcept Exceptionturns that into one printed line and
return None. Becausepost_process_messages_func=Noneis whattells the adapter to miss the cache, the verifier silently degrades into "always call the real LLM" —
the feature added in #669 does nothing, for any client shape.
The repository's own unit test already encodes the intended contract (a
client.chat.completionsobject) and is red on
main:The
hasattr(client, 'completions')probe also picks the wrong branch for aMock(attributes areauto-created) and for
openai>=1.0(which has bothchatandcompletions), and it never matchesthe pre-1.0 module-level
ChatCompletionthatexamples/processor/llm_verifier_example.pyuses.Fix
Route the chat-shaped request to a chat endpoint, keeping both client generations the docs use:
client.chat.completions.createwhen the client exposes.chat(openai>=1.0, langchain-style clients);client.ChatCompletion.createfor the legacy module client;client.completions.createonly as the last resort, as before.No behaviour changes for callers, and the
exceptis left in place so a real API error still degradesto a cache miss rather than raising inside the post-processor.
Verification
Same test file, two trees, base pinned at
c59fb3a6152a4458b2a070ca183b61c4b614095f:cwd
/private/tmp/r16-parent/{base-wt,gptcache},PYTEST_DISABLE_PLUGIN_AUTOLOAD=1,python -m pytest tests/unit_tests/processor/test_post.py -q -o addopts="", base rc=1 / head rc=0,and
gptcache.processor.post.__file__was printed in each run to prove the base run imported theupstream copy, not the patched one.
New tests use hand-written fakes instead of a bare
Mockso the endpoint actually under test isobservable:
_ChatClientsetscompletions = None, so calling it raises instead of silentlyreturning a
Mock, and_LegacyClienthas no.chatat all.Two notes so nobody chases a phantom:
tests/pytest.iniadds--html=... --self-contained-html, soa bare
pytesterrors unlesspytest-htmlis installed or-o addopts=""is used; andtests/unit_tests/processor/test_context.pyfails to collect in my local venv withopenai.lib._old_api.APIRemovedbecause that venv hasopenai>=1.0— that is an environmentartifact, not something this patch touches.
Checklist
bug fix, and the user-visible effect ("semantic verification silently never fires") is stated abovegit difflimited togptcache/processor/post.pyandtests/unit_tests/processor/test_post.py