Refactor databricks-apps around capability composition + warehouse mutations

manifest.json

@@ -46,24 +46,30 @@
       "version": "0.1.0"
     },
     "databricks-apps": {
-      "description": "Build apps on Databricks Apps platform.",
+      "description": "Build AppKit apps on Databricks: compose analytics, Lakebase OLTP/synced reads, Genie, serving, files, jobs, and custom endpoints. Capability-based scaffolding \u2014 invoke BEFORE implementation.",
       "files": [
         "SKILL.md",
         "agents/openai.yaml",
         "assets/databricks.png",
         "assets/databricks.svg",
         "references/appkit/appkit-sdk.md",
         "references/appkit/custom-endpoints.md",
+        "references/appkit/data-patterns.md",
+        "references/appkit/environments.md",
         "references/appkit/files.md",
         "references/appkit/frontend.md",
         "references/appkit/genie.md",
         "references/appkit/jobs.md",
+        "references/appkit/lakebase-oltp.md",
+        "references/appkit/lakebase-synced-reads.md",
         "references/appkit/lakebase.md",
+        "references/appkit/lifecycle.md",
         "references/appkit/model-serving.md",
         "references/appkit/overview.md",
         "references/appkit/proto-contracts.md",
         "references/appkit/proto-first.md",
         "references/appkit/sql-queries.md",
+        "references/appkit/warehouse-mutations.md",
         "references/other-frameworks.md",
         "references/platform-guide.md",
         "references/testing.md"

skills/databricks-apps/references/appkit/custom-endpoints.md

@@ -1,14 +1,18 @@
 # Custom API Endpoints
 
-**CRITICAL**: Do NOT add custom endpoints for SQL queries or warehouse data retrieval. Use `config/queries/` + `useAnalyticsQuery` instead.
+**CRITICAL**: Do NOT add custom endpoints for warehouse **SELECT** queries or read-only data retrieval. Use `config/queries/` + `useAnalyticsQuery` instead.
 
 **CRITICAL**: Do NOT add custom endpoints for Unity Catalog file operations. Use the Files plugin instead.
 
 When you need server-side logic that no plugin covers, extend the AppKit server in `onPluginsReady` and register Express routes with `appkit.server.extend()`.
 
+**Writes are allowed via custom endpoints**, but **which backend** (Postgres app state / Delta DML / Jobs) is decided in **[Data Patterns: Write path](data-patterns.md#write-path)** — the canonical table. Don't re-decide it here.
+
+> **Agentic mode:** never run `databricks apps manifest` or pass `--profile`. For check 2 below, read the enabled plugins from `appkit.plugins.json` / `app.yaml` and the `createApp({ plugins: [...] })` array in `server/server.ts` instead. Checks 1 and 3 apply unchanged. See [Environments](environments.md).
+
 Use custom endpoints ONLY for:
 
-- **Mutations**: Creating, updating, or deleting data (INSERT, UPDATE, DELETE)
+- **Data mutations** — Express routes in `onPluginsReady` using Lakebase or warehouse DML as above (not `useAnalyticsQuery`)
 - **External APIs**: Calling Databricks APIs not covered by a dedicated plugin (MLflow, Workspace API, etc.)
 - **Complex business logic**: Multi-step operations that cannot be expressed in SQL
 - **File processing**: Uploads, processing, transformations (when not covered by the Files plugin)
@@ -44,7 +48,7 @@ databricks apps manifest --profile <PROFILE>
 
 **Key plugins to check for:**
 
-- **analytics** — provides SQL warehouse query execution (do NOT reimplement with custom endpoints)
+- **analytics** — provides SQL warehouse execution: **reads** via `config/queries/` + `useAnalyticsQuery`; **writes** via `appkit.analytics.query()` inside custom mutation routes (see [Warehouse Mutations](warehouse-mutations.md)). Do NOT reimplement SELECT retrieval with custom endpoints.
 - **lakebase** — provides Lakebase plugin for PostgreSQL CRUD (use plugin in routes, don't create raw connections)
 - **genie** — provides Genie AI-powered data exploration (check before building custom natural-language-to-SQL routes)
 - **files** — provides file storage and retrieval helpers (check before writing custom file upload/download routes)
@@ -61,13 +65,23 @@ Read `server/server.ts` to see what routes already exist. Add new handlers insid
 
 ## Server-side Pattern
 
-Register routes inside `onPluginsReady` so plugins are initialized before the server accepts requests:
+Register routes inside `onPluginsReady` so plugins are initialized before the server accepts requests.
+
+**Include the plugins your routes need** — `server()` alone is not enough for warehouse or Lakebase mutations:
+
+| Route purpose | Plugins (minimum) |
+|---------------|-------------------|
+| External Databricks APIs (MLflow, Workspace) | `[server()]` |
+| Delta / UC DML | `[server(), analytics({})]` — see [Warehouse Mutations](warehouse-mutations.md) |
+| Postgres CRUD | `[server(), lakebase()]` — see [Lakebase OLTP](lakebase-oltp.md) |
+| Trigger Lakeflow Jobs | `[server(), jobs()]` — see [Jobs](jobs.md) |
+
+### Example: external API (no warehouse or Lakebase)
 
 ```typescript
 // server/server.ts
 import { createApp, server } from "@databricks/appkit";
 import { getExecutionContext } from "@databricks/appkit";
-import { z } from "zod";
 
 await createApp({
   plugins: [server()],
@@ -82,23 +96,16 @@ await createApp({
         });
         res.json(response);
       });
-
-      // Example: Mutation
-      app.post("/api/records", async (req, res) => {
-        const parsed = z.object({ name: z.string() }).safeParse(req.body);
-        if (!parsed.success) {
-          res.status(400).json({ error: "Invalid input" });
-          return;
-        }
-        // Custom logic here
-        res.status(201).json({ success: true, id: 123 });
-      });
     });
   },
 });
 ```
 
-For Lakebase CRUD routes, schema initialization, and chat persistence, see [Lakebase Guide](lakebase.md).
+Do **not** copy a placeholder mutation that returns fake IDs — real writes must use `appkit.lakebase.query()` or `appkit.analytics.query()` as in [Lakebase OLTP](lakebase-oltp.md) and [Warehouse Mutations Guide](warehouse-mutations.md).
+
+For Lakebase CRUD routes, schema initialization, and chat persistence, see [Lakebase OLTP](lakebase-oltp.md).
+
+For Delta / Unity Catalog writes (`INSERT`, `UPDATE`, `DELETE`, `MERGE`), see [Warehouse Mutations Guide](warehouse-mutations.md).
 
 ## Client-side Pattern
 
@@ -119,43 +126,27 @@ function MyComponent() {
   }, []);
 
   const handleCreate = async () => {
-    await fetch("/api/records", {
+    // POST to your mutation route — see Lakebase or Warehouse Mutations guides for server handlers
+    await fetch("/api/books", {
       method: "POST",
       headers: { "Content-Type": "application/json" },
-      body: JSON.stringify({ name: "test" }),
+      body: JSON.stringify({ title: "Example" }),
     });
   };
 
   return <div>{/* component JSX */}</div>;
 }
 ```
 
-## Decision Tree for Data Operations
-
-1. **Need to display data from SQL?**
-   - **Chart or Table?** → Use visualization components (`BarChart`, `LineChart`, `DataTable`, etc.)
-   - **Custom display (KPIs, cards, lists)?** → Use `useAnalyticsQuery` hook
-   - **Never** add custom endpoints for SQL SELECT statements against the warehouse
-
-2. **Need to call a Databricks API?**
-   - Serving endpoints → use `serving()` plugin (see [Model Serving Guide](model-serving.md))
-   - Jobs → use `jobs()` plugin (see [Jobs Guide](jobs.md))
-   - MLflow, Workspace API, other APIs → custom endpoint via `onPluginsReady`
-
-3. **Need to modify data?** → Custom endpoint in `onPluginsReady`
-   - INSERT, UPDATE, DELETE operations
-   - Multi-step transactions
-   - Business logic with side effects
+## Route patterns and anti-patterns
 
-4. **Need non-SQL custom logic?** → Custom endpoint in `onPluginsReady`
-   - File processing
-   - External API calls
-   - Complex computations in TypeScript
+For **which backend** to use (reads, writes, plugins), see [Data Patterns](data-patterns.md) — do not duplicate that decision tree here.
 
-**Summary:**
+**Summary for custom routes:**
 
-- ✅ SQL queries → Visualization components or `useAnalyticsQuery`
-- ✅ Databricks APIs without a plugin → custom endpoint via `onPluginsReady`
-- ✅ Data mutations → custom endpoint via `onPluginsReady`
-- ❌ SQL warehouse queries → custom endpoints (NEVER do this)
-- ❌ Files operations → custom endpoints (NEVER do this — use Files plugin)
+- ✅ SQL **reads** → `config/queries/` / `useAnalyticsQuery` (not custom endpoints)
+- ✅ Postgres mutations → [Lakebase OLTP](lakebase-oltp.md)
+- ✅ Delta / UC DML → [Warehouse Mutations](warehouse-mutations.md)
+- ✅ Databricks APIs without a plugin → custom endpoint + `onPluginsReady`
+- ❌ Warehouse SELECT in custom endpoints
+- ❌ Files operations in custom endpoints — use [Files](files.md) plugin