Import from sql (#105)

* Update documentation links and prerequisites for SchemaX

- Corrected the directory references from `.schematic/` to `.schemax/` in multiple documentation files to ensure consistency with the current project structure.
- Updated the Python version requirement in the prerequisites section to specify Python 3.11 or newer, reflecting the latest compatibility standards.
- Enhanced the clarity of the setup instructions for initializing a new SchemaX project, ensuring users have accurate guidance for project setup.

* Enhance CLI Import Functionality and Documentation

- Introduced the ability to import from SQL DDL files via the CLI, allowing users to parse and diff against the current state or replace it as a new baseline.
- Updated the CLI documentation to reflect the new import command options, including `--from-sql`, `--mode`, and `--dry-run`.
- Enhanced the workflows documentation to include the new SQL file import process, providing clearer guidance for users.
- Added unit tests for the new import functionality to ensure robustness and correct operation.
- Refactored the import command to handle both live Databricks imports and SQL file imports seamlessly, improving user experience and flexibility.

* Enhance CLI Import Documentation and DDL Parsing

- Updated the CLI documentation for the `--from-sql` option to clarify the supported Unity Catalog DDL statements, including `CREATE`, `ALTER`, and `COMMENT ON`, ensuring users understand the order of operations during import.
- Improved the workflows documentation to reflect the new capabilities of parsing DDL files, emphasizing the ability to handle both `CREATE` and `ALTER` statements in the correct sequence.
- Refactored the DDL parser in the Unity provider to utilize immutable instances for schema, table, and view updates, enhancing the overall robustness and clarity of the code.
- Added detailed comments in the DDL parser to explain the handling of comments and alterations, ensuring maintainability and ease of understanding for future developers.

* Refactor CLI and DDL Parser for Improved Readability and Functionality

- Reformatted the CLI option definitions for better clarity and consistency in the `cli.py` file.
- Enhanced the `import_assets.py` file by consolidating print statements for improved readability.
- Updated the `__init__.py` file to correctly export SQL utility functions, ensuring they are accessible for other modules.
- Refactored the DDL parser in `ddl_parser.py` to improve the handling of comments and schema creation, enhancing the robustness of the parsing logic.
- Cleaned up test files to ensure consistent formatting and readability, particularly in `test_ddl_parser.py` and `test_import_command.py`.
- Removed unnecessary imports in `test_sql_utils.py` to streamline the test suite.

* Update packages/vscode-extension/src/webview/components/ImportAssetsPanel.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update version to 0.2.0 and enhance import functionality

- Bumped version numbers in package.json and pyproject.toml to 0.2.0 for both the Python SDK and VS Code extension.
- Introduced the ability to import from SQL files via the CLI, allowing users to parse and diff against the current state or replace it as a new baseline.
- Added a new "From SQL file" tab in the Import assets modal, enabling users to execute `schemax import --from-sql` without a live Databricks connection.
- Updated documentation to reflect the new import command options and workflows, ensuring users have clear guidance on the new features.

* Enhance DDL Parsing and Testing for Trailing Semicolon Handling

- Updated the DDL parser to strip trailing semicolons from catalog and schema names, ensuring accurate parsing of SQL statements.
- Added unit tests to validate the correct handling of trailing semicolons in `CREATE CATALOG` and `CREATE SCHEMA` commands, addressing previously identified bugs.
- Enhanced the `UnityDDLStateBuilder` to update existing catalogs with comments and tags when created implicitly, improving the robustness of catalog management.
- Improved overall test coverage for DDL parsing functionality, ensuring reliability and correctness in various scenarios.

* fmt

* Enhance Import Functionality and SQL Parsing

- Updated the import functionality in `import_assets.py` to support diff mode with target environments, ensuring that parsed state IDs align with local state to avoid unnecessary drop and recreate operations.
- Introduced catalog mapping updates for environments, allowing for seamless integration of imported assets and improved user experience.
- Enhanced the SQL statement splitting logic in `sql_utils.py` to correctly handle inline comments after semicolons, ensuring accurate parsing of SQL commands.
- Updated the DDL parser documentation to clarify the usage of the dialect parameter, indicating that it is reserved for future use.
- Added unit tests to validate the new behavior of SQL statement parsing, particularly for trailing comments, ensuring robustness in SQL generation and import processes.
- Improved accessibility in the Import Assets Panel by adding ARIA attributes for better screen reader support and user navigation.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Author: vb-dbrks

CONTRIBUTING.md

@@ -3,7 +3,7 @@
 Thank you for your interest in contributing! Full contributing guidelines (code formatting, standards, testing, commit signing, PR process) live in the **documentation site**.
 
 - **Online:** See the [Contributing](https://vb-dbrks.github.io/schemax/contributing) page (after the docs are deployed).
-- **Local:** Run the docs site (`cd docs/schemax && npm run start`) and open [/contributing](http://localhost:3000/schemax-vscode/contributing).
+- **Local:** Run the docs site (`cd docs/schemax && npm run start`) and open [/contributing](http://localhost:3000/schemax/contributing).
 
 Quick links from the Contributing guide:
 

docs/schemax/docs/guide/prerequisites.mdx

@@ -10,11 +10,11 @@ Before you start with SchemaX, ensure you have the following.
 ## For using the VS Code extension
 
 - **VS Code** — A recent version of [Visual Studio Code](https://code.visualstudio.com/) (or Cursor / VS Codium). The extension is developed and tested on current stable releases.
-- **Workspace** — A folder opened in VS Code that will hold (or already holds) your SchemaX project (e.g. a repo with or without an existing `.schematic/` directory).
+- **Workspace** — A folder opened in VS Code that will hold (or already holds) your SchemaX project (e.g. a repo with or without an existing `.schemax/` directory).
 
 ## For using the CLI (apply, validate, SQL, snapshots)
 
-- **Python** — Python 3.9 or newer. The SchemaX CLI and SDK are Python-based.
+- **Python** — Python 3.11 or newer. The SchemaX CLI and SDK are Python-based.
 - **Optional: uv** — [uv](https://docs.astral.sh/uv/) can be used for faster installs and virtual environments; standard `pip` works as well.
 
 ## For building the extension from source (contributors)

docs/schemax/docs/guide/setup.mdx

@@ -48,8 +48,8 @@ schemax --help
 
 ## 4. First-time project init
 
-- If the folder has no `.schematic/` directory, opening the designer (**SchemaX: Open Designer**) will create a new SchemaX project (Unity Catalog by default) with a default environment and empty state.
-- If the folder already has `.schematic/project.json`, SchemaX loads that project.
+- If the folder has no `.schemax/` directory, opening the designer (**SchemaX: Open Designer**) will create a new SchemaX project (Unity Catalog by default) with a default environment and empty state.
+- If the folder already has `.schemax/project.json`, SchemaX loads that project.
 
 ## Next steps
 

docs/schemax/docs/reference/cli.mdx

@@ -37,11 +37,30 @@ Summary of the SchemaX CLI. Run `schemax --help` and `schemax <command> --
 | `schemax apply --target ENV --profile PROFILE --warehouse-id ID [--dry-run] [--no-interaction] [--auto-rollback]` | Execute SQL against target (filtered by env’s deployment scope); records deployment. |
 | `schemax rollback --deployment ID --partial --target ENV --profile P --warehouse-id W` | Undo successful ops from a failed deployment. |
 | `schemax rollback --to-snapshot VERSION --target ENV --profile P --warehouse-id W [--force]` | Complete rollback to a previous snapshot. |
-| `schemax import --target ENV --profile P --warehouse-id W [--catalog NAME] [--adopt-baseline]` | Import existing catalog into changelog; optional baseline. |
+| `schemax import --target ENV --profile P --warehouse-id W [--catalog NAME] [--adopt-baseline]` | Import from **live Databricks**: discover catalog/schema/table and write ops to changelog; optional baseline. |
+| `schemax import --from-sql PATH [--mode diff\|replace] [--dry-run] [--target ENV]` | Import from a **SQL DDL file**: parse file, diff vs current state (or replace as new baseline), write ops to changelog. |
 
 ## Common options
 
 - `--target ENV` — Use environment config (catalog mapping, deployment scope) for SQL generation and apply. See [Environments and deployment scope](/guide/environments-and-scope).
 - `--dry-run` — Preview without executing (apply, rollback).
 - `--no-interaction` — Skip prompts (CI/CD).
 - `--force` — Override baseline guard for complete rollback (use with care).
+
+## Import from SQL file
+
+You can bring existing DDL into SchemaX by pointing the CLI at a `.sql` file instead of a live Databricks workspace:
+
+```bash
+schemax import --from-sql path/to/schema.sql [--mode diff|replace] [--dry-run] [--target ENV]
+```
+
+- **`--from-sql PATH`** — Path to a SQL file containing Unity Catalog DDL (Databricks dialect). **Supported statements** (applied in file order): `CREATE CATALOG`, `CREATE SCHEMA`, `CREATE TABLE`, `CREATE VIEW`, `COMMENT ON`, and **ALTER** — `ALTER TABLE` (ADD COLUMN, DROP COLUMN, RENAME COLUMN, ALTER COLUMN SET NOT NULL / SET DATA TYPE, RENAME TO, SET TBLPROPERTIES) and `ALTER CATALOG` / `ALTER SCHEMA` / `ALTER TABLE` SET TAGS. Statements are applied in the order they appear (e.g. create a table then alter it to add columns or set properties). Unsupported statements are skipped and reported in the summary.
+- **`--mode diff`** (default) — Compare the parsed state to your current project state and append the difference as operations to the changelog. Use this to add objects from the file on top of what you already have.
+- **`--mode replace`** — Treat the parsed file as the new baseline: the diff is computed against an empty state, so the changelog will contain operations that recreate everything in the file. Use this when the SQL file is the single source of truth and you want to replace or bootstrap the project.
+- **`--dry-run`** — Print what would be imported (parsed object counts and planned operations) without writing to the changelog.
+- **`--target ENV`** — Optional; used for catalog mapping consistency when generating SQL later.
+
+**Statement order:** The parser applies each statement in the order it appears in the file. For example, `CREATE TABLE t (id INT);` followed by `ALTER TABLE t ADD COLUMN name STRING;` and `ALTER TABLE t SET TBLPROPERTIES ('k' = 'v');` produces a table with two columns and the property set. This matches the usual script style (create then alter).
+
+If the workspace has no `.schemax/` project yet, `schemax import --from-sql` will create a default Unity Catalog project before importing. In the VS Code extension, use **Import assets** → **From SQL file** tab to pick a file and run the same flow with a UI.

docs/schemax/docs/reference/faq.mdx

@@ -34,7 +34,7 @@ Usually **no**. SchemaX is for people who **define and evolve** catalog/schema/t
 
 ## Is there an audit trail for schema changes?
 
-Yes. All changes are stored as **operations** in `.schematic/changelog.json`, and **snapshots** in `.schematic/snapshots/` capture full state at a point in time. This is versioned in Git, so you have an audit trail of what changed and when. Deployments are also recorded (database-backed when you use `schemax apply`). See [Workflows](/reference/workflows/) and the CLI reference for rollback and deployment tracking.
+Yes. All changes are stored as **operations** in `.schemax/changelog.json`, and **snapshots** in `.schemax/snapshots/` capture full state at a point in time. This is versioned in Git, so you have an audit trail of what changed and when. Deployments are also recorded (database-backed when you use `schemax apply`). See [Workflows](/reference/workflows/) and the CLI reference for rollback and deployment tracking.
 
 ## Can I use the CLI from my own scripts or automation?
 

docs/schemax/docs/reference/workflows.mdx

@@ -179,6 +179,7 @@ sequenceDiagram
 | **Start** | Catalog/schemas/tables already exist in Databricks. Repo may be new or existing. | — |
 | **Init (optional)** | If no project yet: `schemax init`. | `schemax init` |
 | **Import** | Discover live state from provider; diff vs local; generate ops into changelog. | `schemax import --target dev --profile … --warehouse-id … --catalog <name>` (optionally `--dry-run`). |
+| **Import from SQL file** | Parse a DDL file (CREATE + ALTER in file order) and diff vs local (or replace as new baseline); no Databricks connection. | `schemax import --from-sql path/to/schema.sql [--mode diff\|replace] [--dry-run]`. See [CLI Reference — Import from SQL file](/reference/cli#import-from-sql-file). |
 | **Adopt baseline (optional)** | Mark imported state as first deployed for that env: create snapshot, record deployment baseline. | `schemax import … --adopt-baseline` |
 | **From here** | Same as greenfield: design (more ops) → snapshot → apply to dev/test/prod. | Same as Situation 1. |
 

docs/schemax/package.json

@@ -1,6 +1,6 @@
 {
   "name": "schemax-docs",
-  "version": "0.1.4",
+  "version": "0.2.0",
   "private": true,
   "scripts": {
     "docusaurus": "docusaurus",

packages/python-sdk/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## [0.2.0] - 2025-02-12
+
+### Added
+
+- **Import from SQL file**: `schemax import --from-sql PATH [--mode diff|replace] [--dry-run] [--target ENV]` parses a Unity Catalog DDL file and diffs against the current project state (or replaces as new baseline). No Databricks connection required. Statements are applied in file order (e.g. CREATE TABLE then ALTER TABLE ADD COLUMN then SET TBLPROPERTIES).
+- **Core**: `schemax.core.sql_utils.split_sql_statements()` for splitting SQL scripts (preserves quoted semicolons; skips comment-only lines).
+- **Provider contract**: `state_from_ddl(sql_path=..., sql_statements=..., dialect=...)` on the base provider; Unity provider implements full DDL parsing and state building.
+- **Unity DDL parser**: Parses CREATE CATALOG/SCHEMA/TABLE/VIEW, COMMENT ON, and ALTER TABLE (ADD/DROP/RENAME column, ALTER COLUMN, RENAME TO, SET TBLPROPERTIES), ALTER CATALOG/SCHEMA/TABLE SET TAGS. Command-path fallback for CREATE CATALOG/SCHEMA (e.g. when MANAGED LOCATION is present) with comment extraction. State builder uses immutable updates (Pydantic `model_copy`).
+- **Documentation**: CLI reference "Import from SQL file" section; workflows table row; statement-order note.
+
+### Changed
+
+- **Import command**: `schemax import` now supports two sources: live Databricks (requires `--target`, `--profile`, `--warehouse-id`) and SQL file (`--from-sql`). When using `--from-sql`, target/profile/warehouse are optional.
+
 ## [0.1.4] - 2025-02-19
 
 ### Fixed

packages/python-sdk/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "schemaxpy"
-version = "0.1.4"
+version = "0.2.0"
 description = "Python SDK and CLI for Databricks Unity Catalog schema management"
 readme = "README.md"
 requires-python = ">=3.11"

packages/python-sdk/src/schemax/cli.py

@@ -21,6 +21,7 @@
     generate_diff,
     generate_sql_migration,
     import_from_provider,
+    import_from_sql_file,
     rollback_complete,
     validate_project,
 )
@@ -278,10 +279,25 @@ def bundle(target: str, version: str, output: str) -> None:
 
 
 @cli.command(name="import")
-@click.option("--target", "-t", required=True, help="Target environment (dev/test/prod)")
-@click.option("--profile", "-p", required=True, help="Databricks profile name")
-@click.option("--warehouse-id", "-w", required=True, help="SQL warehouse ID")
-@click.option("--catalog", help="Catalog name to import")
+@click.option(
+    "--from-sql",
+    "from_sql_path",
+    type=click.Path(exists=True, path_type=Path),
+    default=None,
+    help="Import from a SQL DDL file instead of live Databricks (optional --target for catalog mapping)",
+)
+@click.option(
+    "--mode",
+    type=click.Choice(["diff", "replace"]),
+    default="diff",
+    help="For --from-sql: diff = append ops to changelog; replace = treat SQL state as new baseline",
+)
+@click.option(
+    "--target", "-t", help="Target environment (required for live import; optional for --from-sql)"
+)
+@click.option("--profile", "-p", help="Databricks profile name (required for live import)")
+@click.option("--warehouse-id", "-w", help="SQL warehouse ID (required for live import)")
+@click.option("--catalog", help="Catalog name to import (live import only)")
 @click.option("--schema", help="Schema name to import (requires --catalog)")
 @click.option("--table", help="Table name to import (requires --catalog and --schema)")
 @click.option(
@@ -298,9 +314,11 @@ def bundle(target: str, version: str, output: str) -> None:
 )
 @click.argument("workspace", type=click.Path(exists=True), required=False, default=".")
 def import_command(
-    target: str,
-    profile: str,
-    warehouse_id: str,
+    from_sql_path: Path | None,
+    mode: str,
+    target: str | None,
+    profile: str | None,
+    warehouse_id: str | None,
     catalog: str | None,
     schema: str | None,
     table: str | None,
@@ -311,10 +329,32 @@ def import_command(
 ) -> None:
     """Import existing provider assets into SchemaX changelog.
 
-    This is a CLI-first import workflow similar to Terraform import:
-    discover live assets -> map to provider state -> generate operations into changelog.
+    Two sources:
+
+    \b
+    • Live Databricks: use --target, --profile, --warehouse-id (and optional scope).
+    • SQL DDL file: use --from-sql path [--mode diff|replace] [--dry-run] [--target ENV].
     """
     try:
+        workspace_path = Path(workspace).resolve()
+
+        if from_sql_path is not None:
+            summary = import_from_sql_file(
+                workspace=workspace_path,
+                sql_path=from_sql_path,
+                mode=mode,
+                dry_run=dry_run,
+                target_env=target,
+            )
+            _print_import_summary(summary)
+            return
+        # Live import: require target, profile, warehouse_id
+        if not target or not profile or not warehouse_id:
+            console.print(
+                "[red]✗[/red] Live import requires --target, --profile, and --warehouse-id. "
+                "Use --from-sql for SQL file import."
+            )
+            sys.exit(1)
         if schema and not catalog:
             console.print("[red]✗[/red] --schema requires --catalog")
             sys.exit(1)
@@ -323,8 +363,6 @@ def import_command(
             sys.exit(1)
         binding_overrides = _parse_catalog_mappings(catalog_map)
 
-        workspace_path = Path(workspace).resolve()
-
         summary = import_from_provider(
             workspace=workspace_path,
             target_env=target,