Skip to content

feat(wren): add context validate with view dry-plan and description checks#1515

Merged
goldmedal merged 4 commits intofeat/cli-0.2.0from
feat/context-validate
Apr 4, 2026
Merged

feat(wren): add context validate with view dry-plan and description checks#1515
goldmedal merged 4 commits intofeat/cli-0.2.0from
feat/context-validate

Conversation

@goldmedal
Copy link
Copy Markdown
Contributor

@goldmedal goldmedal commented Apr 4, 2026

Summary

  • Adds wren context validate CLI command (new context_app sub-application)
  • View SQL dry-plan via wren-core catches broken model references and syntax errors without a DB connection
  • Description completeness warnings for models and views (columns at --level strict)
  • --level flag: error (CI/CD), warning (default), strict (all checks); exit 1 only on errors

Changes

File Change
wren/src/wren/context.py New — validate() core logic + _check_descriptions()
wren/src/wren/context_cli.py New — context_app Typer sub-app with validate command
wren/src/wren/cli.py Register context_app into main CLI
wren/tests/unit/test_context.py 9 unit tests covering all plan scenarios

Test plan

  • test_validate_view_dry_plan_pass — valid view SQL → no errors
  • test_validate_view_dry_plan_error — view referencing deleted model → error
  • test_validate_view_empty_statement — empty statement → error
  • test_validate_model_no_description_warning — model without description → warning
  • test_validate_view_no_description_warning — view without description → warning
  • test_validate_level_error_suppresses_warnings--level error → no warnings
  • test_validate_strict_column_warning--level strict → column warnings
  • test_validate_errors_exit_1 — any error → exit code 1
  • test_validate_warnings_exit_0 — warnings only → exit code 0

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • MDL manifest validation: checks view SQL dry-plans and description completeness with configurable levels (error, warning, strict).
    • New "context" CLI group with a validate command supporting manifest input, datasource/connection options, clear success/error messages, and appropriate exit codes.
  • Tests

    • New unit and CLI tests covering dry-plan failures, empty statements, unknown datasource/level handling, and description-level behaviors.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 4, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: aa7c18cb-4f92-4c5d-9e88-361222d74983

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds MDL manifest validation: new context module with validation logic, a context Typer CLI group and validate command, registers the context subcommand in the root CLI, and adds unit and CLI tests covering dry-plan and description checks.

Changes

Cohort / File(s) Summary
Root CLI
wren/src/wren/cli.py
Registers the new context_app Typer sub-application on the root wren CLI. Also tightens MDL manifest .json file handling to error early if a specified .json path is missing.
Validation core
wren/src/wren/context.py
New module providing validate(manifest_str, data_source, *, level="warning"): base64-decodes JSON manifest, coerces/validates DataSource, instantiates WrenEngine, performs view dry-plan checks, and emits errors/warnings (with optional strict column-description checks).
CLI commands
wren/src/wren/context_cli.py
New Typer app context_app with validate command: accepts --level, --mdl, --datasource, --connection-file; loads manifest, resolves datasource/connection inputs, maps to DataSource, calls context.validate, formats output, and sets exit status based on errors.
Tests
wren/tests/unit/test_context.py
New unit and CLI tests for context.validate and context_cli.validate: dry-plan success/failure, empty-statement handling, description-warning semantics (including strict column checks), invalid datasource/level cases, and CLI exit-code/output assertions.

Sequence Diagram

sequenceDiagram
    actor User
    participant CLI as Root CLI / context_cli.validate()
    participant Context as wren.context.validate()
    participant Engine as WrenEngine
    participant DS as DataSource

    User->>CLI: Invoke `wren context validate` (--mdl, --datasource, --level)
    CLI->>CLI: Load/parse MDL manifest (file / base64)
    CLI->>CLI: Resolve datasource / connection file
    CLI->>DS: Construct/coerce DataSource
    CLI->>Context: Call validate(manifest_str, ds, level)
    Context->>Context: Decode base64 JSON manifest
    Context->>Engine: Instantiate WrenEngine with manifest & datasource
    loop per view
        Context->>Engine: dry_plan(view.statement)
        Engine-->>Context: plan result or raise error
    end
    Context->>Context: Accumulate errors & warnings (incl. descriptions)
    Context-->>CLI: Return {errors, warnings}
    CLI->>User: Print results and exit (code 0 or 1)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • douenergy

Poem

🐇 I nibbled through manifests, base64 and bright,
Checked views, dry-plans, and columns late at night.
Warnings hopped up, errors thumped the ground,
A rabbit gave a check — the context is sound! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 15.79% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically summarizes the main change: adding a new context validate feature with view dry-plan and description checks capabilities.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/context-validate

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@goldmedal goldmedal force-pushed the feat/context-validate branch from c037e7d to 5861c2d Compare April 4, 2026 07:00
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (2)
wren/tests/unit/test_context.py (1)

78-199: Add regression tests for invalid-input contract paths.

Given validate() is a library entry point, it would help to add tests for invalid datasource strings and invalid level values to guarantee structured errors output instead of exceptions.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@wren/tests/unit/test_context.py` around lines 78 - 199, Add two regression
tests that call the library entrypoint validate() with invalid inputs: one
passing an invalid datasource string (e.g., "not-a-datasource") and one passing
an invalid level value (e.g., "nope"); name them
test_validate_invalid_datasource and test_validate_invalid_level. Each test
should call validate(_b64(manifest), <invalid_value>) using the existing
manifest fixtures (e.g., _BASE_MANIFEST or _MODEL_WITHOUT_DESC), assert that
validate returns a structured result (no exception) and that result["errors"] is
non-empty and contains a clear error token like "invalid_datasource" or
"invalid_level" (or other consistent error text your validate() produces).
Ensure tests live alongside the other tests in the same test module and follow
the same pattern (use DataSource.duckdb where appropriate for the valid case to
mirror existing tests).
wren/src/wren/context_cli.py (1)

47-52: Consider moving shared CLI helpers into a dedicated module.

Importing private functions from wren.cli couples two command modules tightly. A small shared utility module would reduce fragility and keep command registration concerns separate from helper logic.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@wren/src/wren/context_cli.py` around lines 47 - 52, The imports of private
helpers (_load_conn, _load_manifest, _require_mdl, _resolve_datasource) from
wren.cli should be relocated to a dedicated shared helper module to avoid
coupling; create a new module (e.g., wren.cli_helpers) that exports these
functions (or thin wrappers) and update context_cli.py to import them from
wren.cli_helpers instead of wren.cli; also update other CLI modules that use
these symbols to import from the new module and add tests or an __init__ export
to ensure public API stability.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@wren/src/wren/context_cli.py`:
- Line 63: The code currently calls _load_manifest(_require_mdl(mdl)) which can
attempt a base64 decode on a non-existent path and produce confusing errors;
update the input validation so that before decoding you explicitly check that
the provided --mdl path exists and is a file (use pathlib.Path(mdl).exists() /
is_file() or os.path.exists) and raise/return a clear error (FileNotFoundError
or a CLI-friendly argparse/Click error) from _require_mdl or at the call site in
context_cli.py so users see "file not found" instead of a decode failure.

In `@wren/src/wren/context.py`:
- Around line 21-22: The f-strings that build guard/warning messages access
model['name'] directly which can raise KeyError for malformed manifests; update
those messages in wren/src/wren/context.py to use model.get("name", "<unknown>")
instead of model['name'] (apply to the occurrences around the current f-strings
and the other instances mentioned), so validation remains resilient and the logs
show "<unknown>" when name is missing.
- Around line 45-46: Validate the level parameter in the functions in
wren/src/wren/context.py that declare level: str = "warning" -> dict: check
against an explicit allowed set (e.g.
{"debug","info","warning","error","critical"}), and if the value is not in that
set return a clear error dict (for example {"error":"invalid level","allowed":
[...]} ) instead of silently proceeding; update both occurrences (the function
with the signature containing level: str = "warning" and the other similar
function referenced in the review) to perform this validation at the start and
return the error dict immediately when invalid.
- Around line 81-83: The check for empty view statements currently treats only
exactly empty strings as empty; update the logic that reads stmt =
view.get("statement", "") in context.py (the variable stmt and list errors) to
trim whitespace (e.g., call strip() on stmt) before testing, so whitespace-only
statements are classified as empty and trigger errors.append(f"View '{name}':
empty statement").
- Around line 74-75: Wrap the DataSource(...) construction in a try/except so
invalid datasource strings don't raise out of the function; when
isinstance(data_source, str) and you call DataSource(data_source), catch the
exception (ValueError/TypeError or general Exception), convert it into the
function's standard validation result (e.g., append/return a validation error
message or ValidationResult indicating invalid datasource) and continue flow
instead of re-raising; reference the DataSource constructor and the local
variable data_source so the change is applied where that conversion occurs.

---

Nitpick comments:
In `@wren/src/wren/context_cli.py`:
- Around line 47-52: The imports of private helpers (_load_conn, _load_manifest,
_require_mdl, _resolve_datasource) from wren.cli should be relocated to a
dedicated shared helper module to avoid coupling; create a new module (e.g.,
wren.cli_helpers) that exports these functions (or thin wrappers) and update
context_cli.py to import them from wren.cli_helpers instead of wren.cli; also
update other CLI modules that use these symbols to import from the new module
and add tests or an __init__ export to ensure public API stability.

In `@wren/tests/unit/test_context.py`:
- Around line 78-199: Add two regression tests that call the library entrypoint
validate() with invalid inputs: one passing an invalid datasource string (e.g.,
"not-a-datasource") and one passing an invalid level value (e.g., "nope"); name
them test_validate_invalid_datasource and test_validate_invalid_level. Each test
should call validate(_b64(manifest), <invalid_value>) using the existing
manifest fixtures (e.g., _BASE_MANIFEST or _MODEL_WITHOUT_DESC), assert that
validate returns a structured result (no exception) and that result["errors"] is
non-empty and contains a clear error token like "invalid_datasource" or
"invalid_level" (or other consistent error text your validate() produces).
Ensure tests live alongside the other tests in the same test module and follow
the same pattern (use DataSource.duckdb where appropriate for the valid case to
mirror existing tests).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7f51d912-6f32-4110-aa63-4632ecc7a4d8

📥 Commits

Reviewing files that changed from the base of the PR and between b086576 and 5861c2d.

📒 Files selected for processing (4)
  • wren/src/wren/cli.py
  • wren/src/wren/context.py
  • wren/src/wren/context_cli.py
  • wren/tests/unit/test_context.py

@goldmedal goldmedal force-pushed the feat/context-validate branch from 5861c2d to 285aa00 Compare April 4, 2026 07:09
@goldmedal goldmedal changed the base branch from main to feat/cli-0.2.0 April 4, 2026 12:31
@goldmedal goldmedal force-pushed the feat/context-validate branch from da32b1f to edc8002 Compare April 4, 2026 13:06
goldmedal and others added 2 commits April 4, 2026 21:15
…hecks

Upgrades `wren context validate` from a structural-only check to a semantic
quality gate with three levels:

- **error** (CI/CD): structural errors + view SQL dry-plan via wren-core
- **warning** (default): + model/view missing description warnings
- **strict**: + column-level missing description warnings

View dry-plan catches broken model references and SQL syntax errors without
touching the data source. Exit code 1 on any error; warnings-only exits 0.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix E402 lint: move context_cli import to top of cli.py
- Use .get("name", "<unknown>") for malformed manifest resilience
- Validate level param in validate() and return error dict if invalid
- Wrap DataSource() construction in try/except for invalid datasource strings
- Strip whitespace from view statement before empty check
- Add test_validate_invalid_datasource and test_validate_invalid_level

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@goldmedal goldmedal force-pushed the feat/context-validate branch from edc8002 to a43dadf Compare April 4, 2026 13:16
_load_manifest silently treated a missing path as a base64 string,
producing a confusing decode error. Now detects .json suffix + missing
file and exits with "Error: MDL file not found: <path>".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@goldmedal goldmedal force-pushed the feat/context-validate branch from a43dadf to 2aafea5 Compare April 4, 2026 13:18
index_manifest now seeds canonical NL-SQL pairs into query_history,
so recall_queries returns seeded results alongside user-stored ones.
Assert >= 1 and verify the user query is present instead of exact count.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@goldmedal goldmedal merged commit 6a8ea38 into feat/cli-0.2.0 Apr 4, 2026
9 checks passed
@goldmedal goldmedal deleted the feat/context-validate branch April 4, 2026 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant