feat(filter): add batch filtering and confidence calibration#25
Merged
foundatron merged 1 commit intomainfrom Mar 11, 2026
Merged
feat(filter): add batch filtering and confidence calibration#25foundatron merged 1 commit intomainfrom
foundatron merged 1 commit intomainfrom
Conversation
Replace per-article filter_article() loop in cli.py with filter_batch(), which sends article batches in a single LLM call. Falls back to individual filter_article() calls on full JSON parse failure or missing/out-of-range entries. Uses 1-based indices in prompts for LLM reliability. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #7
Changes
tentacle/llm/prompts.py— AddFILTER_BATCH_SYSTEMandFILTER_BATCH_USERprompt templatesFILTER_BATCH_SYSTEM: Instructs the LLM to score multiple articles, returning a JSON array of{"index": N, "relevance": 0.XX, "reasoning": "..."}with 1-based indices.FILTER_BATCH_USER: Formats a numbered list of title+abstract pairs ([1] Title: ... / Abstract: ...).tentacle/llm/filter.py— Addfilter_batch()function;filter_article()unchangedfilter_batch(client, articles, *, model, threshold, batch_size=10) -> list[tuple[float, str]]: Chunks articles into batches ofbatch_size, sends each batch in a single LLM call, parses JSON array response.filter_article()individually for every article in that batch.filter_article()for missing ones.max_tokensproportionally (e.g.,batch_size * 100).tentacle/cli.py— Update_run_scan()to usefilter_batch()filter_article()loop with afilter_batch()call overnew_articles, then buildrelevant_articlesfrom the results using the same threshold check.Review Findings
The most impactful fix is #4 (clamp relevance scores in the batch path to match
filter_articlebehavior). #2 (token budget) is a latent reliability issue that will cause silent cost waste. #1 is worth a defensive assertion. The code is otherwise well-structured with solid test coverage and good fallback design.