Skip to content

Add resample pipeline to @itk-wasm/downsample (transform + selectable interpolator)#1548

Open
thewtex wants to merge 10 commits into
InsightSoftwareConsortium:mainfrom
thewtex:resample-pipeline
Open

Add resample pipeline to @itk-wasm/downsample (transform + selectable interpolator)#1548
thewtex wants to merge 10 commits into
InsightSoftwareConsortium:mainfrom
thewtex:resample-pipeline

Conversation

@thewtex

@thewtex thewtex commented Jul 3, 2026

Copy link
Copy Markdown
Member

Summary

Adds a new resample pipeline to @itk-wasm/downsample / itkwasm-downsample. It wraps itk::ResampleImageFilter to resample a moving image onto a reference image's grid, with an optional transform and a selectable interpolator, across the full ITK-Wasm target matrix (WASI, Node, browser/Emscripten, and Python sync + async / WASI + Emscripten).

This was built in five phases: a working C++ prototype, VectorImage support + interpolator ctest coverage, regenerated TypeScript/Python bindings, test data + independent baselines, and the Node/browser/Python test suites.

What changed

C++ pipeline — packages/downsample/resample.cxx (new)

  • Positional input (moving image), reference-image (geometry-only — an empty pixel buffer is accepted, since only origin/spacing/direction/size are read), and output.
  • -t,--transform optional transform mapping output-grid points into moving-image space (defaults to identity).
  • -i,--interpolator selectable from six methods: linear (default), nearest_neighbor, label_image, b_spline, windowed_sinc, gaussian.
  • Supports 2D/3D/4D images over all scalar pixel types and itk::VectorImage (multi-component) types.

Build — packages/downsample/CMakeLists.txt

  • Registers the resample executable and adds the required ITK modules (ITKImageFunction, ITKTransform) plus native transform IO (ITKIOTransformInsightLegacy, ITKIOTransformHDF5).
  • Adds self-contained ctests: a baseline resample smoke test plus one per interpolator (identity resample of cthead1.png), and a label_image test on 2th_cthead1.png — all needing no new test data.

Bindings — regenerated TypeScript + Python

  • TypeScript: resample / resampleNode with ResampleOptions (transform?: TransformList, interpolator?: string) and ResampleResult, exported from the package entry points.
  • Python: sync + async resample across the wasi and emscripten sub-packages.

Test data & baselines

  • New independent baselines and refreshed dam test-data-hash in package.json / test:data:download.

Tests

  • Node: ava tests for the resample bindings.
  • Browser: a demo-app controller + sample-input loader and a Playwright resample.spec.ts.
  • Python: WASI test_resample.py.
  • Adds @itk-wasm/transform-io (TS devDep) and itkwasm-transform-io (pixi) so transform inputs can be exercised in tests.

Why

The downsample package could shrink images but had no general way to map an image onto an arbitrary output grid with a transform and a chosen interpolation method — the standard "resample onto a reference geometry" operation. This adds that as a first-class, fully-bound pipeline.

Implementation notes

  • Transform type: the option is read as itk::wasm::InputTransform<itk::AffineTransform<double, N>> rather than the abstract itk::Transform<double, N, N>. The abstract base won't compile — the memory-IO reader calls TTransform::New(), and itk::Transform has no itkNewMacro. The concrete double-precision AffineTransform is-a Transform<double, N, N> and still feeds SetTransform polymorphically.
  • Geometry-only reference: the filter uses SetReferenceImage(...) + UseReferenceImageOn(), so the reference image contributes only its geometry.
  • VectorImage path: ResampleImageFilter has no native multi-component support, so (mirroring downsample.cxx) each component is extracted (VectorIndexSelectionCastImageFilter), resampled through shared wiring, then recomposed (ComposeImageFilter). Per-component filters are kept alive until a single final compose update, because itk::DataObject only holds a weak pointer back to its producer.
  • Shared wiring: reference-geometry / transform / interpolator setup lives in one MakeResampleFilter<TImage> helper reused by both the scalar and per-component vector paths.

thewtex and others added 7 commits July 3, 2026 11:01
Add packages/downsample/resample.cxx wrapping itk::ResampleImageFilter as a
WASI/Wasm pipeline: a moving image, a reference-image whose geometry defines the
output grid (metadata-only/empty pixel buffer supported), an optional
--transform, and a selectable --interpolator (linear, nearest_neighbor,
label_image, b_spline, windowed_sinc, gaussian) for 2D/3D/4D scalar images. A
shared SelectInterpolator<TImage>() helper returns the common
itk::InterpolateImageFunction<ImageType, double>::Pointer.

Register resample in CMakeLists.txt (foreach list, ITKImageFunction + ITKTransform
components, and native ITKIOTransformInsightLegacy/HDF5) and add a self-contained
ctest that resamples the existing cthead1.png with the default identity transform.

The transform option uses InputTransform<AffineTransform<double, ImageDimension>>
(concrete, double precision) rather than the abstract itk::Transform base, which
cannot compile through the memory-IO reader's TTransform::New().

Verified: pnpm build:wasi (exit 0) and pnpm test:wasi (6/6 pass, incl. resample).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ase 02)

Add an itk::VectorImage specialization to resample.cxx that resamples each
component independently (VectorIndexSelectionCast -> ResampleImageFilter ->
ComposeImageFilter), so every interpolator works with multi-component pixels,
and register the five VariableLengthVector pixel types in SupportInputImageTypes.

Deduplicate the scalar and vector paths behind a shared MakeResampleFilter<T>
helper (reference-geometry/transform/interpolator wiring) alongside the existing
SelectInterpolator<T>. Per-component filters are held alive until a single final
ComposeImageFilter update because DataObject::m_Source is a WeakPointer.

Add ctests exercising each interpolator (linear, nearest_neighbor, b_spline,
windowed_sinc, gaussian) on cthead1.png plus a label_image test on
2th_cthead1.png. WASI build is clean; all 12 ctests pass. Vector-path runtime
validation is deferred to the node/python memory-IO tests in a later phase.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Regenerate the language bindings for the resample pipeline from the
compiled interface metadata (build:emscripten + bindgen:typescript,
build:wasi + bindgen:python).

TypeScript (packages/downsample/typescript/src/):
- Add resample.ts, resample-node.ts, resample-options.ts,
  resample-node-options.ts, resample-result.ts, resample-node-result.ts.
- Re-export resample / ResampleOptions / ResampleResult (+ Node variants)
  from index-only.ts and index-node-only.ts; add the TransformList type
  re-export to index-common.ts.
- Regenerated demo-app scaffolding (resample-controller.ts,
  resample-load-sample-inputs.ts, index.html).

Python:
- Add resample.py (wasi + dispatch) and resample_async.py (emscripten +
  dispatch); each package __init__ exports the symbol.
- Re-embed the emscripten js_package.py bundle; add the wasi test stub.

The wrappers expose the moving Image + reference Image inputs, an optional
transform marshaled as an InterfaceTypes.TransformList memory-IO input
(like affine-ops), and an interpolator string option with the six-value
CLI::IsMember enum. No resample.cxx change was needed and the
@itk-wasm/downsample TS package type-checks cleanly. See
.maestro/.../Working/resample-design-notes.md section 9 for the verified
option shape and a flagged pre-existing core-bindgen quirk.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…h (Phase 04)

Assemble resample test inputs and independent (itk-generated) baselines for the
@itk-wasm/downsample resample pipeline, pack them with dam, and record the new
content hash.

Inputs (test/data/input/, distributed via test/data.tar.gz):
- cthead1-resample-reference.nrrd     scalar reference, different size/spacing/origin
- apple-resample-reference.mha        3-component reference for the vector path
- cthead1-resample-reference-metadata-only.json  geometry-only sidecar (empty-buffer path)
- cthead1-resample-transform.h5       non-identity affine (rotation + translation)

Baselines (test/data/baseline/): cthead1-resample-linear.nrrd,
cthead1-resample-nearest-neighbor.nrrd, apple-resample-linear.mha. Generated with
native itk.ResampleImageFilter wired identically to resample.cxx's MakeResampleFilter,
so they are independent (not self-referential). Verified bit-identical (pxMaxAbs=0)
against the actual resample.wasi.wasm memory-IO output for scalar linear /
nearest_neighbor, the metadata-only empty-buffer reference, and the vector case.

package.json: bump itk-wasm.test-data-hash, the test-data-urls entry, and both hash
occurrences in test:data:download to the new dam CID
(bafkreiesjpg3sjqkyjb77djrffdvznjjsgvudd2y44iqjrfgnd45uyyprq). The repacked
test/data.tar.gz still needs uploading to pinata at the new CID for CI/other machines.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add packages/downsample/typescript/test/node/resample-test.js with four ava
cases exercising resampleNode against the Phase 04 packed inputs and independent
baselines (full node suite: 9/9 pass, no regressions):

- linear + transform: cthead1.png onto cthead1-resample-reference.nrrd with the
  cthead1-resample-transform.h5 affine (read via @itk-wasm/transform-io
  readTransformNode -> Affine/float64/2->2) vs cthead1-resample-linear.nrrd.
- nearest_neighbor + transform: same inputs vs cthead1-resample-nearest-neighbor.nrrd.
- label_image: 2th_cthead1.png as both moving and reference (identity), compared to
  the input itself. itk 5.4.6's wheel lacks LabelImageGenericInterpolateImageFunction
  (GenericLabelInterpolator remote module) so no independent label baseline exists;
  a grid-aligned identity resample is pixel-exact for the label interpolator, mirroring
  the C++ resample-label-image ctest. No new packed baseline / dam re-pack needed.
- VectorImage: apple.jpg + apple-resample-reference.mha with imageType.pixelType
  reassigned to VariableLengthVector (required for the vector dispatch, as
  test_downsample.py's vector test does) vs apple-resample-linear.mha.

Add @itk-wasm/transform-io as a downsample TS devDependency (workspace link) for the
transform reader; pnpm-lock.yaml records the link. common.js is unchanged -- the tests
use its existing testInputPath/testBaselinePath directory exports with inline path.join,
matching every existing node test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ase 05)

Wire the resample function into the browser demo-app, mirroring the
downsample controllers:

- resample-load-sample-inputs.ts: replace the bindgen `export default null`
  stub with a real loader that fetches cthead1.png from the published input
  CID and resamples it onto its own grid (identity, interpolator: linear).
  Self-contained from the single published input; the Phase-04 reference/
  transform files are only in the local data.tar.gz (unpublished raw-tarball
  CID), so they can't be fetched via IPFS. Transform left to its identity
  default; the controller still wires the interactive transform-upload path.
- index.ts: register resample-controller.js (the bindgen-generated controller
  already wires moving/reference/transform/interpolator -> resample; kept as-is
  per the generated-file convention). The resample tab/panel already exist in
  index.html from Phase 03 bindgen.
- vite.config.js: serve @itk-wasm/transform-io pipelines so readTransform's
  hdf5-read-transform wasm loads in-browser (image-io/mesh-io already copied).

Validated: `pnpm build:demo` bundles cleanly (716 modules); dev server serves
/pipelines/resample.*, /pipelines/hdf5-read-transform.*, and downsample.* at
HTTP 200 (no regression).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…(Phase 05)

Completes the resample deliverable — validated C++ -> TypeScript (Node + browser)
-> Python (wasi). No binding/pipeline fixes were needed.

- typescript/test/browser/resample.spec.ts: playwright spec modeled on
  downsample.spec.ts — uploads cthead1.png as both moving + reference (identity
  resample), sets the interpolator, runs, and asserts the output renders.
- python/itkwasm-downsample-wasi/test/test_resample.py: 4 real wasi cases
  (linear+transform, nearest_neighbor+transform, label_image identity,
  VectorImage) compared to the Phase-04 baselines. Reads the .h5 transform via
  itkwasm-transform-io. Placed in test/ (the hand-written suite, next to the real
  test_downsample.py); tests/ (bindgen stubs) is left untouched. Emscripten/
  dispatch stay bindgen stubs, matching every other downsample function.
- pixi.toml/pixi.lock: add itkwasm-transform-io >=1.1.0,<2 (only new dep; images
  via itkwasm-image-io, comparison via itkwasm-compare-images). Lock re-resolved
  (also bumped format v6->v7); consistent with the manifest and CI-safe.

Full matrix green: Node ava 9/9, Python wasi 16 passed, browser playwright 3/3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
thewtex and others added 2 commits July 3, 2026 12:18
Rename the `resample` pipeline to `resample-to-reference` across all
committed source in packages/downsample, applying the correct case per
binding convention (verified against the downsample-label-image
pipeline):

  - kebab   `resample-to-reference`: CMake target + ctests, the
            itk::wasm::Pipeline program name, TS pipelinePath / import
            paths / filenames, wasi wasm-module path
  - camel   `resampleToReference`: TS function names, emscripten
            js_module call, and all demo DOM ids / functionName / tab
            label / notify strings
  - Pascal  `ResampleToReference`: TS Options/Result/Node* types, demo
            Model/Controller classes
  - snake   `resample_to_reference`: Python modules, defs,
            environment_dispatch keys, __init__ re-exports, tests

Descriptive prose, ITK API names (itk::ResampleImageFilter,
ResampleFilterType), "resampled" output filenames, and the dam-managed
test-data filenames are left unchanged. Generated artifacts (dist/,
demo-app/, wasm modules, gitignored tests/ stubs) will refresh on the
next build + bindgen.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The itkwasm-downsample-emscripten js_package.py embeds the compiled
browser worker bundle as a base64 data-url. Regenerate it so the bundle
exports `resampleToReference` (and the `resample-to-reference` pipeline
path) to match the rename; the emscripten async wrapper calls
`js_module.resampleToReference(...)`, which the pre-rename bundle (only
exporting `resample`) would not have resolved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@thewtex

thewtex commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

@sedghi @vboussot please take a look

This adds a pipeline to resample according to the sampling grid defined by a reference image. Another PR will add a pipeline to resample according to direct specification of the grid.

…o/tests

The resample-to-reference demo controller and Node test import
`@itk-wasm/transform-io` (readTransform / readTransformNode), but
`@itk-wasm/downsample-build` did not depend on `@itk-wasm/transform-io-build`.
CI builds a package via `pnpm --filter "<pkg>-build..." build:gen:typescript`,
whose `...` closure is what produces each sibling io package's `dist/`. With
transform-io absent from that closure, its `dist/` was never built, so the
demo's `vite build` failed to resolve the package entry:

  [commonjs--resolver] Failed to resolve entry for package "@itk-wasm/transform-io"

Add `@itk-wasm/transform-io-build` to downsample-build's devDependencies,
mirroring the existing `@itk-wasm/image-io-build` /
`@itk-wasm/compare-images-build` entries, so the build closure now includes
transform-io (verified: `pnpm --filter "@itk-wasm/downsample-build..."` now
lists transform-io). Also add `@itk-wasm/transform-io` to the demo
vite.config `optimizeDeps.exclude`, matching image-io/mesh-io — it is now a
runtime-loaded sibling pipeline (as in the transform and mesh-filters
packages).

Reproduced the failure locally by hiding transform-io/dist and confirmed the
demo build resolves once its dist is present.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sedghi

sedghi commented Jul 3, 2026

Copy link
Copy Markdown

I thought this would be a new /resample package, but it looks like you're putting it under downsample. Is that intentional?


These are the things Fable found

Findings (most severe first)

  1. packages/downsample/resample-to-reference.cxx:170 — The --transform option is bound to InputTransform<itk::AffineTransform<double, N>> while the public API advertises a generic TransformList, and the deserializer never checks the transform parameterization. WasmTransformToTransformFilter's non-composite path validates only precision and dimensions, then raw-copies parameters into the affine. A Euler2DTransform from @itk-wasm/transform-io (3 params, dims 2→2, float64) passes every guard and silently produces a wrongly-resampled image; a BSpline transform (more params than the affine's N×(N+1)) makes std::copy write past the parameter buffer — wasm heap corruption; a composite list hits a null dynamic_cast and crashes; a multi-element non-composite list silently keeps only the last entry. Using itk::CompositeTransform<double, N> as the input type (as the core transform-read-write test pipeline does) would deserialize any factory-registered parameterization correctly and support chains.
  2. packages/downsample/python/itkwasm-downsample-wasi/pyproject.toml:36 — All three Python packages keep itkwasm >= 1.0.b145, but the new bindings unconditionally do from itkwasm import ... TransformList at import time, and TransformList only exists in itkwasm ≥ 1.0b180. An environment holding itkwasm 1.0b145–b179 (which pip considers satisfied) gets ImportError on import itkwasm_downsample, breaking every pre-existing function, not just the new one. Same stale floor in itkwasm-downsample/pyproject.toml:37 and itkwasm-downsample-emscripten/pyproject.toml:36; bump to ≥ 1.0b180 (or transform-io's b185).
  3. packages/downsample/python/itkwasm-downsample-wasi/itkwasm_downsample_wasi/resample_to_reference.py:73 — Interpolator validation is interpolator not in ('linear,nearest_neighbor,...') — a single comma-joined string, so in does substring matching, not membership. interpolator='near', 'sinc', or 'gauss' passes the check and dies deep in the wasm pipeline with an opaque CLI error instead of the intended ValueError. Root cause is in bindgen (wasi-function-module.js:237 splits choices on ', ' but they serialize without spaces) — this PR is the first artifact to hit that path, so fix the generator and regenerate.
  4. packages/downsample/resample-to-reference.cxx:157 — The reference image is declared as InputImage using the moving image's exact pixel/component type, but the help text (propagated into every TS/Python docstring) promises "only the metadata is used, so an empty pixel buffer is acceptable." A routine cross-type call — float32 moving + uint8 reference — throws "Unexpected component type" at deserialization. The metadata-only claim is also never tested: the test-data tarball even packs cthead1-resample-reference-metadata-only.json "to exercise the empty-buffer path," yet nothing references it, and in Python data=None marshals through np.asarray(None).tobytes() into garbage bytes. Either relax the reference type, or fix the docs and exercise the fixture.
  5. packages/downsample/resample-to-reference.cxx:90 — SelectInterpolator ends in a bare else that silently returns linear for unknown names, and the six-name interpolator list is hand-maintained in three places (the two CLI::IsMember lists at lines 183 and 254 plus this chain). Unreachable today because CLI11 gates input, but any future drift (a seventh name added to the CLI lists only) silently yields linear-interpolated output instead of an error — invisible to the run-success-only tests. Prefer a single name→factory table and an itkExceptionMacro on unknown names.
  6. packages/downsample/typescript/README.md:26 — The bindgen-generated README (the published npm docs for @itk-wasm/downsample) was not regenerated: it documents the five pre-existing functions but has no resampleToReference/resampleToReferenceNode section, even though all other TS artifacts were regenerated. The new API ships invisible on npm, and the next full bindgen run will produce a surprise README diff.
  7. packages/downsample/resample-to-reference.cxx:228 — The VectorImage functor re-declares the scalar functor's entire ~35-line option surface verbatim, including the duplicated CLI::IsMember list and multi-sentence help strings. Since dispatch picks a functor by input pixel type at runtime, drift between the copies would fork the CLI surface between scalar and vector inputs silently. This exceeds the package convention (downsample.cxx duplicates ~11 trivial lines with no enum constraint); hoist the declarations into a templated helper alongside MakeResampleFilter.
  8. packages/downsample/CMakeLists.txt:80 — The resample-to-reference smoke ctest is argument-for-argument identical to resample-to-reference-linear (same executable, same inputs, same explicit --interpolator linear; only the output basename differs) — pure duplicate CI work. Additionally, none of the six per-interpolator tests compares output against anything (exit-code-0 only), so a broken interpolator that still writes an image passes the whole matrix. Drop the duplicate and consider a foreach() with a baseline comparison.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants