Make Jev semantic checks best effort (#75)

Jev checks now surface anything they cannot evaluate as warnings that name
the skipped comment or file and why, instead of failing. `nudge check` exits
1 only for error findings and deterministic violations; exit 2 is removed.
Uncertain judgments are warnings for both warn and block rules.

- Retry 429, 5xx, and connection failures up to three times with jittered
  exponential backoff (200/400/800 ms), bounded by the deadline.
- Send batches through an 8-worker pool; a failed request skips only its own
  batch, and a malformed answer skips only its own comment.
- Judge comments without following code against the preceding code or the
  enclosing block instead of failing the whole file; skip comments with no
  code in scope.
- Bump tree-sitter-rust to 0.24.2, which parses `&raw` borrows of a variable
  named `raw`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Author: jssblck

.agents/skills/nudge/references/ci.md

@@ -17,7 +17,6 @@ Exit behavior:
 
 - `0`: no checkable violations were found, or no file-based rules exist.
 - `1`: one or more checkable violations were found.
-- `2`: uncertain or incomplete semantic checks; takes precedence over findings.
 - Other non-zero exits can indicate configuration, argument, or runtime errors.
 
 With explicit operands, every path or glob must resolve to at least one file.
@@ -48,8 +47,9 @@ It does not evaluate live-hook surfaces:
 If the user wants CI coverage, prefer file-content rules for conventions that
 must be enforceable outside a live agent.
 
-Semantic warnings exit 0; error/block findings exit 1. Uncertain or incomplete
-semantic scans exit 2, taking precedence over errors.
+Semantic error/block findings exit 1. Warnings, uncertain judgments, and skipped
+semantic checks exit 0; skipped checks print a warning naming what was not
+evaluated.
 
 ## GitHub Actions Example
 

.agents/skills/nudge/references/hook-responses.md

@@ -80,7 +80,8 @@ For noisy, silent, or surprising rules, read
 
 Opt-in Rust-comment rules choose `action: warn` or `action: block`. Completed
 findings at or above `thresholds.violation` warn or block according to that action.
-Uncertain judgments and incomplete checks always allow the tool with model-visible
+Uncertain judgments and skipped checks always allow the tool with model-visible
 context. Treat uncertainty as a question to review, not an established violation.
-Missing credentials or service errors do not establish that code is clean.
+A skipped check names the comment or file that was not evaluated and why; it does
+not establish that the code is clean.
 Deterministic blocks still take priority, and warnings preserve substitutions.

.agents/skills/nudge/references/rule-writing.md

@@ -523,10 +523,12 @@ rules:
 Add an equivalent `tool: Edit` matcher to check edits. On Edit, Nudge evaluates
 comments whose text or adjacent code changed in the resulting file. It does not
 interpret a replacement fragment as a complete file. Ambiguous replacements,
-missing source files, invalid Rust syntax, or comments without associated code
-produce incomplete-check warnings. Consecutive comments are grouped; doc comments
-and comment-like strings are excluded. Rust fences in Markdown are supported with
-`target: { kind: MarkdownCodeBlock, language: rust }`.
+missing source files, or invalid Rust syntax skip the file with a warning.
+Consecutive comments are grouped; doc comments and comment-like strings are
+excluded. A comment is judged with the code after it, the code before it when it
+closes a block, or its enclosing block when that block has no other code.
+Comments with no code in scope are not evaluated. Rust fences in Markdown are
+supported with `target: { kind: MarkdownCodeBlock, language: rust }`.
 
 Optional `content` (Write) or `new_content` (Edit) matchers act as deterministic
 preconditions on each selected target. With semantic Edit rules these conditions
@@ -535,28 +537,35 @@ apply to the resulting file or code block, not just the replacement string.
 Choose `action: warn` for warning-level findings or `action: block` for
 error-level findings that prevent the hook operation. Values at or below `clear`
 pass; values at or above `violation` trigger the chosen action. Values between
-them report uncertainty and allow the operation. API failures and incomplete
-checks also allow the operation with a warning, even for `action: block`.
+them are uncertain and reported as warnings, even for `action: block`.
 
 `thresholds.violation` is the probability that the violation statement is true,
 not a separate confidence score. For example, `violation: 0.95` triggers at 95%
 or higher. The thresholds must satisfy `0 <= clear < violation <= 1`. Choose
 thresholds for each rule based on your code and tolerance for false positives.
 Semantic judgments are probabilistic; selecting `block` does not make them facts.
 
-The client pins `jev-1.13.0`, batches independent judgments, and uses a one-second
-semantic deadline for a hook invocation. It does not retry failures or follow
-HTTP redirects. Missing credentials, rate limiting, timeout, oversized context,
-or malformed responses produce explicit incomplete warnings. Check mode has a
-30-second network-evaluation budget for the scan. Source selection and Jev's
-network call are separate from deterministic checks; existing block rules still
-take priority in hooks.
-
-`nudge check` exits 0 for a complete scan with no errors, including warning-only
-findings; 1 for error-level semantic findings or deterministic violations; and 2
-for uncertain or incomplete semantic checks. Incomplete or uncertain takes
-precedence over errors, and never prints an all-clear result. Warnings are printed
-but do not fail CI.
+Semantic checks are best effort. The client pins `jev-1.13.0`, batches
+independent judgments, and sends up to eight batches concurrently. It retries
+rate limits (429), server errors including overload (5xx), and connection
+failures up to three times with jittered exponential backoff (about 200, 400,
+and 800 ms), never past the deadline. It does not retry other failures or follow
+HTTP redirects. Hooks have a one-second semantic deadline; `nudge check` has 30
+seconds for the whole scan.
+
+When a check cannot run, Nudge reports a skipped-check warning naming the file,
+line, and rule it did not evaluate, and why: for example, missing credentials,
+exhausted retries, timeout, oversized context, or a malformed answer. Failures
+are isolated. A failed request skips only its own batch, and a malformed answer
+skips only its own comment. Skipped checks never block a hook or fail CI; they
+mean Nudge cannot vouch for that code, not that it is clean.
+
+Source selection and Jev's network call are separate from deterministic checks;
+existing block rules still take priority in hooks.
+
+`nudge check` exits 1 for error-level semantic findings or deterministic
+violations and 0 otherwise. Warnings, uncertain judgments, and skipped checks
+are printed, with a count of skipped checks, but do not fail CI.
 
 Validate configuration offline with `nudge validate`. For a sample evaluation:
 

.claude/skills/nudge/references/ci.md

@@ -17,7 +17,6 @@ Exit behavior:
 
 - `0`: no checkable violations were found, or no file-based rules exist.
 - `1`: one or more checkable violations were found.
-- `2`: uncertain or incomplete semantic checks; takes precedence over findings.
 - Other non-zero exits can indicate configuration, argument, or runtime errors.
 
 With explicit operands, every path or glob must resolve to at least one file.
@@ -48,8 +47,9 @@ It does not evaluate live-hook surfaces:
 If the user wants CI coverage, prefer file-content rules for conventions that
 must be enforceable outside a live agent.
 
-Semantic warnings exit 0; error/block findings exit 1. Uncertain or incomplete
-semantic scans exit 2, taking precedence over errors.
+Semantic error/block findings exit 1. Warnings, uncertain judgments, and skipped
+semantic checks exit 0; skipped checks print a warning naming what was not
+evaluated.
 
 ## GitHub Actions Example
 

.claude/skills/nudge/references/hook-responses.md

@@ -80,7 +80,8 @@ For noisy, silent, or surprising rules, read
 
 Opt-in Rust-comment rules choose `action: warn` or `action: block`. Completed
 findings at or above `thresholds.violation` warn or block according to that action.
-Uncertain judgments and incomplete checks always allow the tool with model-visible
+Uncertain judgments and skipped checks always allow the tool with model-visible
 context. Treat uncertainty as a question to review, not an established violation.
-Missing credentials or service errors do not establish that code is clean.
+A skipped check names the comment or file that was not evaluated and why; it does
+not establish that the code is clean.
 Deterministic blocks still take priority, and warnings preserve substitutions.

.claude/skills/nudge/references/rule-writing.md

@@ -523,10 +523,12 @@ rules:
 Add an equivalent `tool: Edit` matcher to check edits. On Edit, Nudge evaluates
 comments whose text or adjacent code changed in the resulting file. It does not
 interpret a replacement fragment as a complete file. Ambiguous replacements,
-missing source files, invalid Rust syntax, or comments without associated code
-produce incomplete-check warnings. Consecutive comments are grouped; doc comments
-and comment-like strings are excluded. Rust fences in Markdown are supported with
-`target: { kind: MarkdownCodeBlock, language: rust }`.
+missing source files, or invalid Rust syntax skip the file with a warning.
+Consecutive comments are grouped; doc comments and comment-like strings are
+excluded. A comment is judged with the code after it, the code before it when it
+closes a block, or its enclosing block when that block has no other code.
+Comments with no code in scope are not evaluated. Rust fences in Markdown are
+supported with `target: { kind: MarkdownCodeBlock, language: rust }`.
 
 Optional `content` (Write) or `new_content` (Edit) matchers act as deterministic
 preconditions on each selected target. With semantic Edit rules these conditions
@@ -535,28 +537,35 @@ apply to the resulting file or code block, not just the replacement string.
 Choose `action: warn` for warning-level findings or `action: block` for
 error-level findings that prevent the hook operation. Values at or below `clear`
 pass; values at or above `violation` trigger the chosen action. Values between
-them report uncertainty and allow the operation. API failures and incomplete
-checks also allow the operation with a warning, even for `action: block`.
+them are uncertain and reported as warnings, even for `action: block`.
 
 `thresholds.violation` is the probability that the violation statement is true,
 not a separate confidence score. For example, `violation: 0.95` triggers at 95%
 or higher. The thresholds must satisfy `0 <= clear < violation <= 1`. Choose
 thresholds for each rule based on your code and tolerance for false positives.
 Semantic judgments are probabilistic; selecting `block` does not make them facts.
 
-The client pins `jev-1.13.0`, batches independent judgments, and uses a one-second
-semantic deadline for a hook invocation. It does not retry failures or follow
-HTTP redirects. Missing credentials, rate limiting, timeout, oversized context,
-or malformed responses produce explicit incomplete warnings. Check mode has a
-30-second network-evaluation budget for the scan. Source selection and Jev's
-network call are separate from deterministic checks; existing block rules still
-take priority in hooks.
-
-`nudge check` exits 0 for a complete scan with no errors, including warning-only
-findings; 1 for error-level semantic findings or deterministic violations; and 2
-for uncertain or incomplete semantic checks. Incomplete or uncertain takes
-precedence over errors, and never prints an all-clear result. Warnings are printed
-but do not fail CI.
+Semantic checks are best effort. The client pins `jev-1.13.0`, batches
+independent judgments, and sends up to eight batches concurrently. It retries
+rate limits (429), server errors including overload (5xx), and connection
+failures up to three times with jittered exponential backoff (about 200, 400,
+and 800 ms), never past the deadline. It does not retry other failures or follow
+HTTP redirects. Hooks have a one-second semantic deadline; `nudge check` has 30
+seconds for the whole scan.
+
+When a check cannot run, Nudge reports a skipped-check warning naming the file,
+line, and rule it did not evaluate, and why: for example, missing credentials,
+exhausted retries, timeout, oversized context, or a malformed answer. Failures
+are isolated. A failed request skips only its own batch, and a malformed answer
+skips only its own comment. Skipped checks never block a hook or fail CI; they
+mean Nudge cannot vouch for that code, not that it is clean.
+
+Source selection and Jev's network call are separate from deterministic checks;
+existing block rules still take priority in hooks.
+
+`nudge check` exits 1 for error-level semantic findings or deterministic
+violations and 0 otherwise. Warnings, uncertain judgments, and skipped checks
+are printed, with a count of skipped checks, but do not fail CI.
 
 Validate configuration offline with `nudge validate`. For a sample evaluation:
 

AGENTS.md

@@ -138,12 +138,12 @@ When Nudge has something to share, it responds in one of several ways:
 - **Continue**: For UserPromptSubmit hooks, Nudge injects context as plain text
 - **Learned context**: For UserPromptSubmit hooks, Nudge searches `.nudge/learned/*.md` with BM25, or hybrid BM25 plus local embeddings when `learn.embeddings.enabled` is set in `.nudge.yaml` or `.nudge.yml`, and injects the most relevant incident notes when the prompt resembles a known scenario. For supported PreToolUse command surfaces, learned context can be surfaced as an allow-with-context warning.
 - **Interrupt**: For PreToolUse hooks, Nudge blocks the operation and explains what to fix
-- **Warning**: For uninspectable provider inputs and opt-in Jev semantic findings, uncertainty, or incomplete checks, Nudge allows the operation and returns model-visible context.
+- **Warning**: For uninspectable provider inputs and opt-in Jev semantic findings, uncertainty, or skipped checks, Nudge allows the operation and returns model-visible context.
 - **Substitute**: For deterministic PreToolUse Bash rules, Nudge rewrites the command and lets it proceed
 
 The response type is determined by the hook type:
 - `PreToolUse` block rules **interrupt** (block provider-supported Write/Edit/WebFetch/Bash operations)
-- `PreToolUse` semantic rules choose `warn` (**allow with context**) or `block` (**deny completed findings**). Uncertain or incomplete checks always allow with context.
+- `PreToolUse` semantic rules choose `warn` (**allow with context**) or `block` (**deny completed findings**). Uncertain judgments and skipped checks always allow with context.
 - `PreToolUse` substitute rules **allow with updated input** (Claude Code, Codex CLI, Grok Build, and Cursor Bash commands)
 - `UserPromptSubmit` rules always **continue** (inject guidance into the conversation)
 - `PermissionRequest` is parsed but always **passes through** until Nudge has a permission-specific rule surface
@@ -192,7 +192,7 @@ client reads `TYPESAFE_API_KEY`, falling back to user-level `credentials.json`
 saved by `nudge login typesafe.ai`, and uses a fixed TypeSafe endpoint.
 `src/semantic.rs` owns planning and policy; `src/semantic/` contains configuration,
 selection, exact edit snapshots, and the HTTP client. Tests inject a transport,
-without reading credentials or contacting Jev. Hooks allow uncertain/incomplete
-checks with warnings. Check mode exits 0 for warning-only findings, 1 for errors,
-and 2 for uncertain or incomplete semantic scans.
+without reading credentials or contacting Jev. Checks are best effort: retryable
+failures back off and retry, and uncertain judgments or checks that cannot run are
+warnings naming what was skipped. Only error findings block hooks or fail `nudge check`.
 See [the semantic rules guide](docs/semantic-rules.md) for the full contract.

CLAUDE.md

@@ -126,12 +126,12 @@ When Nudge has something to share, it responds in one of several ways:
 - **Continue**: For UserPromptSubmit hooks, Nudge injects context as plain text
 - **Learned context**: For UserPromptSubmit hooks, Nudge searches `.nudge/learned/*.md` with BM25, or hybrid BM25 plus local embeddings when `learn.embeddings.enabled` is set in `.nudge.yaml` or `.nudge.yml`, and injects the most relevant incident notes when the prompt resembles a known scenario. For supported PreToolUse command surfaces, learned context can be surfaced as an allow-with-context warning.
 - **Interrupt**: For PreToolUse hooks, Nudge blocks the operation and explains what to fix
-- **Warning**: For uninspectable provider inputs and opt-in Jev semantic findings, uncertainty, or incomplete checks, Nudge allows the operation and returns model-visible context.
+- **Warning**: For uninspectable provider inputs and opt-in Jev semantic findings, uncertainty, or skipped checks, Nudge allows the operation and returns model-visible context.
 - **Substitute**: For deterministic PreToolUse Bash rules, Nudge rewrites the command and lets it proceed
 
 The response type is determined by the hook type:
 - `PreToolUse` block rules **interrupt** (block provider-supported Write/Edit/WebFetch/Bash operations)
-- `PreToolUse` semantic rules choose `warn` (**allow with context**) or `block` (**deny completed findings**). Uncertain or incomplete checks always allow with context.
+- `PreToolUse` semantic rules choose `warn` (**allow with context**) or `block` (**deny completed findings**). Uncertain judgments and skipped checks always allow with context.
 - `PreToolUse` substitute rules **allow with updated input** (Claude Code, Codex CLI, Grok Build, and Cursor Bash commands)
 - `UserPromptSubmit` rules always **continue** (inject guidance into the conversation)
 - `PermissionRequest` is parsed but always **passes through** until Nudge has a permission-specific rule surface
@@ -180,7 +180,7 @@ client reads `TYPESAFE_API_KEY`, falling back to user-level `credentials.json`
 saved by `nudge login typesafe.ai`, and uses a fixed TypeSafe endpoint.
 `src/semantic.rs` owns planning and policy; `src/semantic/` contains configuration,
 selection, exact edit snapshots, and the HTTP client. Tests inject a transport,
-without reading credentials or contacting Jev. Hooks allow uncertain/incomplete
-checks with warnings. Check mode exits 0 for warning-only findings, 1 for errors,
-and 2 for uncertain or incomplete semantic scans.
+without reading credentials or contacting Jev. Checks are best effort: retryable
+failures back off and retry, and uncertain judgments or checks that cannot run are
+warnings naming what was skipped. Only error findings block hooks or fail `nudge check`.
 See [the semantic rules guide](docs/semantic-rules.md) for the full contract.

Cargo.lock

@@ -21,9 +21,8 @@ nudge check "**/*.rs"
 
 Exit behavior:
 
-- `0`: no errors were found (semantic warnings are allowed), or no file-based rules exist.
+- `0`: no errors were found (semantic warnings, uncertain judgments, and skipped checks are allowed), or no file-based rules exist.
 - `1`: one or more deterministic violations or semantic errors were found.
-- `2`: a semantic check was uncertain or incomplete; this takes precedence over errors.
 - Other non-zero exits: configuration, argument, or runtime errors.
 
 With no path arguments, `nudge check` scans the whole project from the current
@@ -49,11 +48,11 @@ Checked 25 files against 6 rules
 
 Semantic Rust-comment rules with `action: warn` or `action: block` also run in check
 mode. Use a saved credential from `nudge login typesafe.ai`, or `TYPESAFE_API_KEY`
-for CI. Selected source context is sent to TypeSafe. Warning findings exit 0;
-error/block findings exit 1; uncertain or incomplete semantic evaluation exits 2
-and takes precedence over other findings. Check mode never prints an all-clear
-result for those states. See [semantic rules](semantic-rules.md) for thresholds,
-budgets, and setup. Static `nudge validate` remains offline.
+for CI. Selected source context is sent to TypeSafe. Semantic checks are best
+effort: error/block findings exit 1, while warnings, uncertain judgments, and
+checks skipped for missing credentials or service failures exit 0 with a warning
+that names what was not evaluated. See [semantic rules](semantic-rules.md) for
+thresholds, retries, budgets, and setup. Static `nudge validate` remains offline.
 
 `nudge check` loads the same rule files as hook mode: