docs: document Sigima snapshot and release workflow

Refs #18

Assisted-by: GPT-5.6 Sol

Author: PierreRaybaut

.env.template

@@ -2,14 +2,15 @@
 #
 # Copy this file to ``.env`` and adjust ``PYTHONPATH`` to point at your
 # local sibling checkouts of Sigima and guidata (or remove those entries
-# entirely if you installed them with ``pip install -r requirements-dev.txt``
-# and want to use the wheels from PyPI).
+# entirely if you used ``scripts/sigima_dependency.py install`` and want to
+# exercise the manifest-selected packages instead). Sibling paths always take
+# priority over packages installed in the virtual environment.
 #
 # The Python interpreter itself is the project-local virtualenv created
 # with::
 #
 #     py -3.11 -m venv .venv
-#     .\.venv\Scripts\python -m pip install -r requirements-dev.txt
+#     .\.venv\Scripts\python scripts\sigima_dependency.py install
 #
 # ``scripts/run_with_env.py`` automatically picks up ``.venv\Scripts\python``
 # when no ``PYTHON`` override is set. Set ``PYTHON=...`` here only if you
@@ -21,7 +22,8 @@
 
 PYTHONPATH=.;..\Sigima;..\guidata
 
-# Pyodide cannot import sibling source trees through PYTHONPATH. To exercise
-# unreleased Sigima changes in the browser, build a wheel under ../Sigima/dist
-# and point Vite at it with an absolute /@fs/ URL (use forward slashes):
+# Pyodide cannot import sibling source trees through PYTHONPATH. During rapid
+# local iteration, build the sibling checkout under ../Sigima/dist and point
+# Vite at it with an absolute /@fs/ URL (use forward slashes). This local value
+# overrides both fields in sigima-dependency.json for browser runtimes only:
 # VITE_SIGIMA_INSTALL_SPEC=/@fs/C:/path/to/Sigima/dist/sigima-X.Y.Z-py3-none-any.whl

.vscode/tasks.json

@@ -466,7 +466,7 @@
     },
     {
       "label": "📦 Release Package (app + SDK)",
-      "detail": "Lint + Vitest + build, then produce datalab-web-<v>.tgz and datalab-platform-web-sdk-<v>.tgz under release/.",
+      "detail": "Sigima release guard + lint + Vitest + qualified build, then produce both tarballs under release/.",
       "type": "shell",
       "command": "npm",
       "args": ["run", "release:pack"],

CONTRIBUTING.md

@@ -81,6 +81,34 @@ DataLab-Web is fully internationalised: English is the source language and Frenc
 
 DataLab-Web sometimes backports a feature or patches a bug that is fixed upstream (`guidata`, `sigima`, …) but not yet in a released wheel. These **temporary shims** are tracked centrally so they can be audited and removed once upstream catches up. Every backport shim is declared once in [src/runtime/shims/registry.ts](src/runtime/shims/registry.ts), carries `# TEMPORARY SHIM` / `@shim-registry: <id>` markers in its source, and is kept in sync by a network-free anti-drift test that runs in `npm test`. Use `npm run audit:shims` for the fast PyPI/lockfile pre-audit and `npm run audit:shims:runtime` to verify the versions actually installed in Pyodide. A `ready-to-remove` version result is only a candidate: focused contract or E2E tests must also prove native behavioral parity before the shim is deleted. The full workflow is in [doc/shim-registry.md](doc/shim-registry.md).
 
+## Sigima development snapshots
+
+When DataLab-Web needs a Sigima change that is merged but not yet published,
+declare it in [sigima-dependency.json](sigima-dependency.json). Keep
+`publishedRequirement` as the exact release target (`sigima==X.Y.Z`) and set
+`developmentRef` to the full lowercase 40-character commit SHA. Branch names,
+tags, abbreviated SHAs and version ranges are rejected because they are not an
+immutable, release-qualified dependency.
+
+Do not add a source URL to [requirements-dev.txt](requirements-dev.txt) or copy
+the SHA into a workflow. The following command installs the common Python
+requirements and then resolves the configured Sigima selection:
+
+```powershell
+.\.venv\Scripts\python scripts\sigima_dependency.py install
+```
+
+The `PYTHONPATH` entries in `.env` take priority over installed packages;
+remove `..\Sigima` when the purpose of the test is to qualify the exact SHA
+declared in the manifest.
+
+CI uses the same resolver to build the Pyodide wheel. A non-null
+`developmentRef` blocks `release:guard`, `release:pack`, the tag helper and the
+release workflow. After the exact target version is published, qualify CPython
+and Pyodide without `PYTHONPATH` or `VITE_SIGIMA_INSTALL_SPEC`, then set
+`developmentRef` to `null`. See [doc/releasing.md](doc/releasing.md) for the
+release checklist.
+
 ## Branching model
 
 DataLab-Web follows the same two-branch model as the sibling repositories (DataLab, Sigima): day-to-day work lands on **`develop`**, and **`main`** is the release branch — `develop` is merged into `main` only when cutting a real release. CI ([tests.yml](.github/workflows/tests.yml)) runs the cheap regression suite on both branches and on pull requests targeting either. The multi-minute performance benchmarks are **not** part of that run; they are opt-in and driven by a separate on-demand workflow (see the **Performance benchmarks** section of [doc/testing-strategy.md](doc/testing-strategy.md)).

README.md

@@ -64,10 +64,25 @@ The first dev load downloads Pyodide (~10 MB) and installs Sigima via `micropip`
 
 ### Testing unreleased Sigima changes
 
-`PYTHONPATH` makes the sibling Sigima checkout available to the CPython tests,
-but the browser's Pyodide runtime requires a wheel. To test DataLab-Web against
-changes on Sigima's `develop` branch before they are released, first build the
-sibling checkout:
+Cross-repository integration snapshots are declared once in
+[`sigima-dependency.json`](sigima-dependency.json). `publishedRequirement` is
+the exact PyPI version qualified for releases; `developmentRef` is either
+`null` or a full 40-character commit SHA already merged into Sigima. The test
+and performance workflows turn that immutable SHA into both the CPython
+requirement and the Pyodide wheel. Install the same configured dependency
+locally with:
+
+```powershell
+.\.venv\Scripts\python scripts\sigima_dependency.py install
+```
+
+If `.env` still includes `..\Sigima` in `PYTHONPATH`, that sibling checkout
+intentionally takes priority for CPython. Remove the entry when qualifying the
+exact manifest-selected snapshot.
+
+For rapid work on an uncommitted sibling checkout, `PYTHONPATH` makes the local
+Sigima sources available to CPython, but the browser still requires a wheel.
+Build that checkout directly:
 
 ```powershell
 cd ..\Sigima
@@ -85,8 +100,15 @@ VITE_SIGIMA_INSTALL_SPEC=/@fs/C:/Dev/Sigima/dist/sigima-X.Y.Z-py3-none-any.whl
 Replace `X.Y.Z` with the generated filename, then use the usual `npm run dev`
 or Playwright commands. The override applies to the main runtime and the macro
 and notebook workers. Rebuild the wheel and restart Vite after each Sigima
-change. Remove the line to return to the released PyPI requirement; `/@fs/`
-URLs are for local development only and must not be used for release builds.
+change. This ignored `.env` override takes priority over the published
+requirement but does not change the versioned CI snapshot. Remove the line to
+return to the exact PyPI requirement; `/@fs/` URLs are for local development
+only and must not be used for release builds.
+
+Any non-null `developmentRef` deliberately blocks DataLab-Web releases. Once
+the target Sigima version is published and qualified without a local override,
+set the field to `null`; the generalized snapshot mechanism remains available
+for the next coordinated change.
 
 ## Documentation
 

doc/releasing.md

@@ -2,21 +2,40 @@
 
 ## Versioning
 
-The application version is declared **once**, in `package.json`, and is injected into the bundle at build time via Vite's `define` option (see `vite.config.ts`). The _Help → About_ dialog reads it from `import.meta.env.VITE_APP_VERSION`.
+The application version is declared in `package.json`, kept aligned with `packages/sdk/package.json`, and injected into the bundle at build time via Vite's `define` option (see `vite.config.ts`). The _Help → About_ dialog reads it from `import.meta.env.VITE_APP_VERSION`.
 
-To bump the version, use the standard npm command (it edits `package.json`, creates a commit, and tags it `vX.Y.Z`):
+Use the release helper to bump both packages, promote the changelog, create one commit and tag it `vX.Y.Z`:
 
 ```powershell
-npm version patch   # bug fix:  0.1.0 → 0.1.1
-npm version minor   # feature:  0.1.0 → 0.2.0
-npm version major   # breaking: 0.1.0 → 1.0.0
+node scripts/release.mjs patch   # bug fix:  0.1.0 → 0.1.1
+node scripts/release.mjs minor   # feature:  0.1.0 → 0.2.0
+node scripts/release.mjs major   # breaking: 0.1.0 → 1.0.0
 ```
 
-The next `npm run dev` or `npm run build` automatically picks up the new value — no other file needs to be edited.
+The push remains manual, providing a final checkpoint before the release workflow starts.
 
-> **Keep `packages/sdk/package.json` in sync** — bump its `version` to the same value before tagging. The release CI fails if the two `package.json` files disagree.
+## Sigima release qualification
 
-> **What `git push --tags` triggers** — the [`Release tarballs`](../.github/workflows/release.yml) workflow runs, in order: version coherence check (tag ↔ both `package.json` files) → `pytest tests/python` (3.11 + 3.12) and Playwright E2E (in parallel) → lint + Vitest + build + pack the two `.tgz` → publish a GitHub Release with the tarballs and auto-generated notes → deploy `dist/` to GitHub Pages. Any failing gate aborts the release **and** the deploy.
+[`sigima-dependency.json`](../sigima-dependency.json) separates the exact
+published requirement from the temporary integration snapshot. Before
+preparing a DataLab-Web release:
+
+1. Confirm that `publishedRequirement` is an exact pin such as
+   `sigima==1.3.0` and is available on PyPI.
+2. Run pytest and Playwright without a sibling Sigima `PYTHONPATH`, local wheel
+   or `VITE_SIGIMA_INSTALL_SPEC`.
+3. Confirm the live Pyodide runtime reports that exact Sigima version.
+4. Set `developmentRef` to `null` only after those checks pass.
+
+`npm run release:guard` performs the network-free structural checks. It blocks
+while any snapshot SHA is active and is called before `release:pack`, before
+the release helper mutates files, and by the release workflow before its other
+jobs. The release Python tests explicitly install `publishedRequirement`, and
+the release browser tests never build or export a development wheel.
+`release:pack` uses `build:release`, whose release mode ignores even an
+accidental `VITE_SIGIMA_INSTALL_SPEC` from the developer's ignored `.env`.
+
+> **What `git push --tags` triggers** — the [`Release tarballs`](../.github/workflows/release.yml) workflow runs, in order: Sigima release guard and version coherence check (tag ↔ both `package.json` files) → `pytest tests/python` (3.11 + 3.12) and Playwright E2E against the published Sigima pin (in parallel) → lint + Vitest + build + pack the two `.tgz` → publish a GitHub Release with the tarballs and auto-generated notes → deploy `dist/` to GitHub Pages. Any failing gate aborts the release **and** the deploy.
 
 ## Distribution: app bundle + SDK tarballs
 
@@ -30,7 +49,7 @@ DataLab-Web is shipped to integrators as **two `.tgz` artefacts** produced by th
 Generate them locally:
 
 ```powershell
-npm run release:pack   # lint → test → build → SDK pack → app pack → summary
+npm run release:pack   # guard → lint → test → build → SDK pack → app pack
 ```
 
 Or invoke each step independently (`npm run sdk:pack`, `npm run app:pack`). Output lands in `release/`.

doc/testing-strategy.md

@@ -28,7 +28,7 @@ A continuous layer ties the three together: GitHub Actions (`tests.yml`) runs al
 # introspection).
 Copy-Item .env.template .env
 py -3.11 -m venv .venv
-.\.venv\Scripts\python -m pip install -r requirements-dev.txt
+.\.venv\Scripts\python scripts\sigima_dependency.py install
 
 # Python unit tests + coverage report (htmlcov-python/)
 .\.venv\Scripts\python -m pytest tests/python --cov=src/runtime --cov-report=html:htmlcov-python
@@ -49,6 +49,20 @@ npx playwright test --project=perf
 $env:PW_REPRO=1; npx playwright test --project=repro tests/e2e/_repro_x.spec.ts
 ```
 
+The resolver reads [sigima-dependency.json](../sigima-dependency.json). Normal
+test and performance CI use its `developmentRef` when one is configured;
+release CI explicitly selects `publishedRequirement`. The browser receives a
+development snapshot only as a wheel through `VITE_SIGIMA_INSTALL_SPEC`.
+For rapid local iteration, the ignored `.env` may point to a wheel built from
+the sibling checkout, and its `PYTHONPATH` may prioritize sibling Python
+sources, but neither alters the versioned CI selection. Remove those local
+overrides when qualifying the exact manifest-selected dependency.
+
+When qualifying a published Sigima version for release, use a clean environment
+without `..\Sigima` in `PYTHONPATH` and without `VITE_SIGIMA_INSTALL_SPEC`.
+This ensures both pytest and Playwright exercise the exact PyPI version rather
+than a sibling checkout or cached local wheel.
+
 Test layout:
 
 ```text