Fix standalone localStorage WAL bloat: Rust per-window session store (#225)

Author: nedtwigg

docs/specs/standalone.md

@@ -64,7 +64,9 @@ stderr, which Rust appends to the log file). Webview → Rust is the Tauri
 `pty_get_scrollback` / `get_available_shells`, `dor_control_response`,
 `iframe_create_proxy_url`, the `agent_browser_*` family, the `clipboard`
 readers, `read_update_log`, and `kill_sidecar_now` — each a thin forwarder to
-the corresponding sidecar message, with two carve-outs: on Windows the
+the corresponding sidecar message. `load_session` / `save_session` are the
+exception that is *not* forwarded: they read/write the per-window session file
+directly in Rust (§Persistence). Two further carve-outs: on Windows the
 clipboard readers skip the sidecar and read the Win32 clipboard natively
 (`clipboard_win.rs`; behavior in `docs/specs/mouse-and-clipboard.md` §8.6),
 and `agent_browser_screenshot` receives a temp-file *path* from the sidecar
@@ -169,13 +171,59 @@ The workspace strip lands here when the workspaces rollout reaches stage 3 —
 
 ## Persistence
 
-`TauriAdapter.saveState` / `getState` store the session blob in webview
-`localStorage` under `TauriAdapter.STATE_KEY`, routed through
+`TauriAdapter.saveState` / `getState` route the session blob through
 `lib/src/lib/window-persistence.ts` (`loadSessionState` / `saveSessionState`)
 — the standalone adapter boundary where the `PersistedWindow` wrapping lives,
 identity-passthrough while the workspaces flag is off
-(`docs/specs/transport.md`, Workspace/Window containers). Theme selection
-persists separately through the theme store (`docs/specs/theme.md`).
+(`docs/specs/transport.md`, Workspace/Window containers). The backing store is
+**not** WebKit `localStorage`: `window-persistence.ts` reads/writes through the
+`SessionKeyValueStore` seam, and the standalone adapter supplies a Rust-backed
+implementation (`standalone/src/tauri-session-store.ts`). Theme selection still
+persists through the theme store on `localStorage` (`docs/specs/theme.md`); it
+is tiny and rarely written, so it does not stress the WebKit store.
+
+**Why not `localStorage`.** WKWebView stores `localStorage` as SQLite in WAL
+mode. Dormouse rewrites the multi-MB scrollback-bearing session blob on every
+save, and WebKit pins its own WAL with a long-lived reader that never advances
+during a running session — so the WAL is never checkpointed and grows unbounded
+(observed ~1 GB after a few hours; an external checkpoint is blocked by the same
+reader). A days-long session made this pathological.
+
+**Rust file store.** `save_session(window, state)` / `load_session(window)`
+(`lib.rs`) persist the blob as one atomic file per Tauri window —
+`<app_data_dir>/sessions/<label>.json`, written temp-then-rename so a crash
+cannot truncate the previous snapshot. There is no WAL to grow, and overwriting
+in place bounds the on-disk size to one blob. **Window identity is implicit**:
+each command keys by the invoking `tauri::Window`'s `label()`, so the frontend
+stays window-agnostic and a second window (`win-2`, …) persists to its own file
+without ever rewriting the first window's blob — the store is multi-window even
+though the app ships a single window today.
+
+**Boot + the synchronous-read constraint.** `getState()` is synchronous because
+cold-start restore reads it before React mounts, but a Tauri `invoke` is async.
+`TauriSessionStore` resolves this with an in-memory write-through cache: `init()`
+(awaited by `bootstrap()` before `resumeOrRestore`) hydrates the cache from
+`load_session`, `getItem` returns the cache synchronously, and `setItem` updates
+the cache and forwards to `save_session` asynchronously, coalescing bursts to at
+most one in-flight write (latest value wins). This mirrors how the VS Code
+adapter reads a host-injected seed (`docs/specs/vscode.md`).
+
+**Migration.** On the first boot after this change `load_session` returns null;
+if a legacy blob is still in `localStorage` under `TauriAdapter.STATE_KEY`, the
+adapter adopts it, persists it to the Rust store (through the store's normal
+write path, so it shares the coalescing), and removes the key — so WebKit stops
+rewriting it and its bloated WAL collapses on the next quit.
+
+**Durability on quit (current limitation).** `saveState` returns after updating
+the cache and *firing* `save_session`; nothing awaits the Rust write on
+shutdown — the `onRequestSessionFlush` handshake is a no-op here (VS Code-only),
+and the normal quit path does not intercept the window close (`updater.ts`
+`onCloseRequested` only prevents default for a pending update). So a clean quit
+can drop the save fired at `pagehide`, losing state changed in the final
+debounce/heartbeat window — a regression from the old `localStorage` path, which
+WebKit flushed on teardown. Accepted for now because restore is best-effort and
+saves are frequent (≤500 ms for layout changes); a drain-on-quit is planned
+(`## Future`).
 
 ## File drop
 
@@ -224,13 +272,14 @@ root `package.json` for the `dev:standalone*` orchestration.
 
 | File | Role |
 |------|------|
-| `standalone/src-tauri/src/lib.rs` | Rust backend: sidecar spawn/supervision, invoke commands, event forwarding, file drop, logging, dock icon, exit teardown |
+| `standalone/src-tauri/src/lib.rs` | Rust backend: sidecar spawn/supervision, invoke commands, event forwarding, per-window session file store (`save_session` / `load_session`), file drop, logging, dock icon, exit teardown |
 | `standalone/src-tauri/src/clipboard_win.rs` | Native Win32 clipboard reads on Windows (owned by `docs/specs/mouse-and-clipboard.md`) |
 | `standalone/scripts/tauri.mjs`, `csp.mjs` | Tauri CLI wrapper assembling the webview CSP (`DORMOUSE_REMOTE_CONNECT_SRC`) |
 | `standalone/src-tauri/tauri.conf.json` | Window config, dev/build commands, sidecar resources glob, updater config |
 | `standalone/src/main.tsx` | Webview bootstrap (boot sequence above) |
 | `standalone/src/AppBar.tsx` | Titlebar: shell dropdown, theme picker, window controls |
-| `standalone/src/tauri-adapter.ts` | `TauriAdapter`: PlatformAdapter over Tauri invoke/events, localStorage persistence, control-request dispatch |
+| `standalone/src/tauri-adapter.ts` | `TauriAdapter`: PlatformAdapter over Tauri invoke/events, session persistence via the Rust store, control-request dispatch |
+| `standalone/src/tauri-session-store.ts` | `TauriSessionStore`: Rust-backed `SessionKeyValueStore` — boot-seeded write-through cache over `load_session` / `save_session` (§Persistence) |
 | `standalone/src/updater.ts`, `UpdateBanner.tsx`, `UpdateDebugModal.tsx` | Auto-update (owned by `docs/specs/auto-update.md`) |
 | `standalone/src/browser-sidecar-host.ts`, `browser-sidecar-adapter.ts` | Browser-dev harness (owned by `docs/specs/transport.md`) |
 | `standalone/sidecar/main.js` | Sidecar entry: stdio JSON-lines dispatch, shutdown ordering, parent-PID watchdog |
@@ -239,3 +288,14 @@ root `package.json` for the `dev:standalone*` orchestration.
 | `standalone/sidecar/clipboard-ops.js` | OS clipboard tiers (owned by `docs/specs/mouse-and-clipboard.md`) |
 | `standalone/scripts/build-sidecar-proxy.mjs` | Bundles `lib/src/host/` into the sidecar `.cjs` copies |
 | `standalone/scripts/dev-agent-browser.mjs` | `dev:standalone:ab` entry (owned by `docs/specs/transport.md`) |
+
+## Future
+
+**Drain-on-quit for the session store.** Restore the last-save durability the
+`localStorage` path had before the Rust store (§Persistence, "Durability on
+quit"). Add `TauriSessionStore.drain()` — resolves when no `save_session` is
+in-flight or pending — and have the close path (`updater.ts` `onCloseRequested`)
+`preventDefault`, run a final `flushSessionSave()`, `await` the drain (with a
+timeout so a hung write can't wedge quit), then close. Lands with the part-2
+save-path rework (write-on-change + heartbeat removal), which reshapes the same
+path.

docs/specs/transport.md

@@ -114,7 +114,7 @@ Source of truth: `lib/src/lib/session-types.ts` defines the persisted-session in
 
 **Workspace/Window containers (implemented, dormant behind the `dormouse.flags.workspaces` flag; rollout ledger in `docs/specs/layout.md` `## Future`).** A **Workspace** persists as a `PersistedWorkspace`: a `WorkspaceId`, a user-facing `name`, and the Workspace's `PersistedSession` (its panes, doors, and dockview layout). The standalone Window persists as a `PersistedWindow`: the ordered list of `PersistedWorkspace` plus the active `WorkspaceId`. Source of truth: `PersistedWorkspace` / `PersistedWindow` / `readPersistedWindow` / `replaceActiveSession` in `session-types.ts`. VS Code does **not** use `PersistedWindow`; each webview persists exactly one `PersistedSession` — its single Workspace — through the same per-surface state API as today (`workspaceState` for the view, `vscode.setState()` per editor panel; see `docs/specs/vscode.md`).
 
-The wrapping lives at the **standalone adapter boundary**, not in the shared save/restore code: `lib/src/lib/window-persistence.ts` (`activeSessionFromStored` / `storedValueForSession`) translates between the host's stored top-level blob and the bare `PersistedSession` that `reconnect.ts` / `session-save.ts` operate on, and `tauri-adapter.ts` / `browser-sidecar-adapter.ts` route `getState` / `saveState` through it. With the flag **off** (the default) these are identity passthroughs — the stored blob stays a bare `PersistedSession` and behavior is byte-identical to pre-workspace behavior. With the flag **on**, load returns the active Workspace's session and save merges it back into the active slot, preserving the other Workspaces.
+The wrapping lives at the **standalone adapter boundary**, not in the shared save/restore code: `lib/src/lib/window-persistence.ts` (`activeSessionFromStored` / `storedValueForSession`) translates between the host's stored top-level blob and the bare `PersistedSession` that `reconnect.ts` / `session-save.ts` operate on, and `tauri-adapter.ts` / `browser-sidecar-adapter.ts` route `getState` / `saveState` through it. The blob round-trips through a `SessionKeyValueStore` — a single synchronous key/value slot the host persists natively: the browser-dev sidecar uses `localStorage`, while the real standalone adapter uses a Rust-backed per-window file store (`docs/specs/standalone.md` §Persistence), never WebKit `localStorage`. With the flag **off** (the default) these are identity passthroughs — the stored blob stays a bare `PersistedSession` and behavior is byte-identical to pre-workspace behavior. With the flag **on**, load returns the active Workspace's session and save merges it back into the active slot, preserving the other Workspaces.
 
 Versioning and migration: the standalone top-level snapshot is a `PersistedWindow` (its own `version: 1`) wrapping v3 sessions. A pre-workspace bare `PersistedSession` (any version) migrates on read to a single `PersistedWorkspace` named `Workspace 1`, marked active, inside a `PersistedWindow` (`readPersistedWindow`); unreadable inner sessions are dropped and a dangling `activeWorkspaceId` is repaired to the first Workspace. A host that hands back a bare `PersistedSession` (VS Code, or legacy/flag-off standalone storage) is read as one Workspace. Migrations stay additive — older shapes keep flowing v1→v2→v3→(window) without losing panes, doors, alert state, or surface kind.
 

lib/src/lib/window-persistence.ts

@@ -42,20 +42,34 @@ export function storedValueForSession(existingStored: unknown, session: unknown)
   return existingWindow ? replaceActiveSession(existingWindow, next) : wrapSessionInWindow(next);
 }
 
+/**
+ * The seam below the shared save/restore code: a single synchronous key/value
+ * slot the host persists natively. `localStorage` (browser-dev sidecar) and the
+ * standalone `TauriSessionStore` (a Rust-backed, boot-seeded cache) both satisfy
+ * it — the same interface, two host-native backings (`docs/specs/standalone.md`
+ * §Persistence). `Storage` is a structural superset, so passing `localStorage`
+ * still type-checks. VS Code does not go through here; it persists one bare
+ * `PersistedSession` per webview through the extension host's own state APIs.
+ */
+export interface SessionKeyValueStore {
+  getItem(key: string): string | null;
+  setItem(key: string, value: string): void;
+}
+
 // Storage-level round trip shared by the standalone adapters (Tauri + the
-// browser-dev sidecar). Owns the JSON parse/stringify and the `Storage` access
+// browser-dev sidecar). Owns the JSON parse/stringify and the store access
 // so each adapter's get/save collapses to one call instead of re-implementing
 // the read-merge-write dance.
 
 /** Read the stored blob and return the `PersistedSession` to restore (or null). */
-export function loadSessionState(storage: Storage, key: string): unknown {
+export function loadSessionState(storage: SessionKeyValueStore, key: string): unknown {
   const raw = storage.getItem(key);
   if (raw === null) return null;
   return activeSessionFromStored(JSON.parse(raw));
 }
 
 /** Persist `session` under `key`, merging into the active Workspace when the flag is on. */
-export function saveSessionState(storage: Storage, key: string, session: unknown): void {
+export function saveSessionState(storage: SessionKeyValueStore, key: string, session: unknown): void {
   // Flag off (the default): store the bare session without reading the existing
   // blob — its previous value is irrelevant, so skip parsing the (potentially
   // large, scrollback-bearing) stored snapshot.

standalone/src-tauri/src/lib.rs

@@ -572,6 +572,78 @@ fn read_update_log() -> Result<String, String> {
     read_log_tail(10_000)
 }
 
+// --- Per-window session persistence (docs/specs/standalone.md §Persistence) ---
+//
+// The webview's persisted-session blob (a `PersistedWindow`) is stored as one
+// atomic file per Tauri window, keyed by the window label. This replaces webview
+// `localStorage`, whose WKWebView SQLite WAL grew unbounded because WebKit pins
+// its own WAL with a long-lived reader and never truncates during a days-long
+// session. A plain file we overwrite atomically has no WAL and cannot grow.
+//
+// Window identity is implicit: each command keys by the invoking window's label,
+// so the frontend stays window-agnostic and a second window (`win-2`, …) persists
+// to its own file without ever rewriting the first window's blob.
+
+fn sessions_dir(app: &AppHandle) -> Result<PathBuf, String> {
+    Ok(app
+        .path()
+        .app_data_dir()
+        .map_err(|e| format!("app_data_dir unavailable: {e}"))?
+        .join("sessions"))
+}
+
+// Window labels are app-controlled (e.g. "main"), but sanitize defensively so a
+// label can never escape the sessions directory or embed a path separator.
+fn session_file_name(label: &str) -> String {
+    let safe: String = label
+        .chars()
+        .map(|c| {
+            if c.is_ascii_alphanumeric() || c == '-' || c == '_' {
+                c
+            } else {
+                '_'
+            }
+        })
+        .collect();
+    format!("{safe}.json")
+}
+
+fn read_session_from(dir: &Path, label: &str) -> Result<Option<String>, String> {
+    let path = dir.join(session_file_name(label));
+    match std::fs::read_to_string(&path) {
+        Ok(contents) => Ok(Some(contents)),
+        Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(None),
+        Err(e) => Err(format!("read session {label}: {e}")),
+    }
+}
+
+fn write_session_to(dir: &Path, label: &str, state: &str) -> Result<(), String> {
+    create_dir_all(dir).map_err(|e| format!("create sessions dir: {e}"))?;
+    let file_name = session_file_name(label);
+    let path = dir.join(&file_name);
+    let tmp = dir.join(format!("{file_name}.tmp"));
+    // Atomic replace: write a sibling temp file, fsync it, then rename over the
+    // target so a crash mid-write can never truncate the previous good snapshot.
+    {
+        let mut f = File::create(&tmp).map_err(|e| format!("open temp: {e}"))?;
+        f.write_all(state.as_bytes())
+            .map_err(|e| format!("write temp: {e}"))?;
+        f.sync_all().map_err(|e| format!("fsync temp: {e}"))?;
+    }
+    std::fs::rename(&tmp, &path).map_err(|e| format!("rename session {label}: {e}"))?;
+    Ok(())
+}
+
+#[tauri::command]
+fn load_session(window: tauri::Window) -> Result<Option<String>, String> {
+    read_session_from(&sessions_dir(window.app_handle())?, window.label())
+}
+
+#[tauri::command]
+fn save_session(window: tauri::Window, state: String) -> Result<(), String> {
+    write_session_to(&sessions_dir(window.app_handle())?, window.label(), &state)
+}
+
 #[tauri::command]
 fn kill_sidecar_now(state: tauri::State<'_, SidecarState>) {
     kill_sidecar_and_wait(&state.child);
@@ -1103,6 +1175,8 @@ pub fn run() {
             read_clipboard_image_as_file_path,
             read_clipboard_text,
             read_update_log,
+            load_session,
+            save_session,
             agent_browser_command,
             agent_browser_edit,
             agent_browser_screenshot,
@@ -1129,8 +1203,8 @@ pub fn run() {
 #[cfg(test)]
 mod tests {
     use super::{
-        find_node_binary, resolve_dor_cli_paths, resolve_sidecar_path,
-        strip_windows_verbatim_prefix,
+        find_node_binary, read_session_from, resolve_dor_cli_paths, resolve_sidecar_path,
+        session_file_name, strip_windows_verbatim_prefix, write_session_to,
     };
     use std::fs;
     use std::path::{Path, PathBuf};
@@ -1343,4 +1417,47 @@ mod tests {
         assert_eq!(resolved, sidecar_path);
         assert!(!resolved.to_string_lossy().contains(r"\\?\"));
     }
+
+    #[test]
+    fn session_missing_reads_none() {
+        let dir = TempDir::new("sessions-missing");
+        // No file yet — a fresh install / new window reads as None, not an error.
+        assert_eq!(read_session_from(dir.path(), "main").unwrap(), None);
+    }
+
+    #[test]
+    fn session_round_trips_and_isolates_windows() {
+        let dir = TempDir::new("sessions-roundtrip");
+        write_session_to(dir.path(), "main", r#"{"v":1,"who":"main"}"#).unwrap();
+        assert_eq!(
+            read_session_from(dir.path(), "main").unwrap().as_deref(),
+            Some(r#"{"v":1,"who":"main"}"#),
+        );
+
+        // A second window persists to its own file and never touches the first's.
+        write_session_to(dir.path(), "win-2", r#"{"v":1,"who":"win-2"}"#).unwrap();
+        assert_eq!(
+            read_session_from(dir.path(), "main").unwrap().as_deref(),
+            Some(r#"{"v":1,"who":"main"}"#),
+        );
+        assert_eq!(
+            read_session_from(dir.path(), "win-2").unwrap().as_deref(),
+            Some(r#"{"v":1,"who":"win-2"}"#),
+        );
+
+        // Overwrite is atomic-replace, not append: the latest blob fully wins.
+        write_session_to(dir.path(), "main", r#"{"v":2}"#).unwrap();
+        assert_eq!(
+            read_session_from(dir.path(), "main").unwrap().as_deref(),
+            Some(r#"{"v":2}"#),
+        );
+    }
+
+    #[test]
+    fn session_label_cannot_escape_directory() {
+        // A hostile label is flattened to a plain filename inside the dir.
+        assert_eq!(session_file_name("../../evil"), "______evil.json");
+        assert_eq!(session_file_name("main"), "main.json");
+        assert_eq!(session_file_name("a/b"), "a_b.json");
+    }
 }