Make shapely an optional [vector] extra (#2496) (#2497)

* Make shapely an optional [vector] extra (#2496)

shapely (and GEOS) loaded on every import xrspatial because rasterize.py
imported it at module top level. Import it lazily via a cached
_require_shapely() helper, bound locally in each function that uses the
shapely array API (so dask tile workers, which call the helpers directly,
also get the friendly error). rasterize() calls it up front for an early,
clear failure.

Remove shapely>=2.0 from install_requires; add a vector extra and keep
shapely in the tests extra. polygonize already imports shapely lazily and
only on its geopandas return path, so it needs no change. Update README,
install docs, and CHANGELOG.

* Address review nit: also assert polygonize imports without shapely (#2496)

Author: brendancol

CHANGELOG.md

@@ -45,6 +45,7 @@ for the per-tier semantics and the audit trail.
 
 
 #### Bug fixes and improvements
+- Move shapely from a required dependency to an optional `vector` extra. shapely (and GEOS) used to load on every `import xrspatial` because `rasterize.py` imported it at module top level. It is now imported lazily, so a plain `pip install xarray-spatial` neither installs nor loads shapely. The only paths that need it are the vector-to-raster functions `rasterize` and `polygonize`; install `pip install xarray-spatial[vector]` to use them. Calling `rasterize` without shapely raises a clear ImportError pointing at the extra. Follows the same pattern as the matplotlib change (#2494). (#2496)
 - Move matplotlib from a required dependency to an optional `plot` extra. A plain `pip install xarray-spatial` no longer pulls in matplotlib (and its pillow / fonttools / kiwisolver / contourpy / cycler / pyparsing chain), since no compute path imports it -- every matplotlib use is a lazy import inside the `.xrs.plot` accessor helpers. Install `pip install xarray-spatial[plot]` to use plotting; calling `.xrs.plot()` without it now raises a clear ImportError pointing at the extra instead of a bare `ModuleNotFoundError`. pandas stays required because xarray depends on it and several core modules import it directly. (#2494)
 - Reconcile `xrspatial.geotiff.SUPPORTED_FEATURES` with the GeoTIFF release-contract tiering proposed in epic #2340. Adds `reader.windowed` at `stable` (covered by the existing window-read suite) and `reader.dask` at `stable` (covered by the cross-backend parity matrix in `test_backend_parity_matrix.py` and `test_backend_full_parity_2211.py`). Demotes `reader.allow_rotated` and `reader.allow_unparseable_crs` from `advanced` to `experimental` to match the epic's placement of permissive read-side escape hatches in the Experimental tier. A new shape test (`test_supported_features_shape_2348.py`) pins the structural invariants of the mapping (every entry carries a tier label; the tier set is closed at `{stable, advanced, experimental, internal_only}`; the dict literal contains no duplicate keys) so future drift fails CI. Runtime behaviour of every read and write path is unchanged; this is metadata-only. Callers gating on the exact string value of these two demoted entries will need to update their checks. (#2348)
 - Promote the local COG read and write paths to the `stable` tier in `xrspatial.geotiff.SUPPORTED_FEATURES`. `SUPPORTED_FEATURES['writer.cog']` and `SUPPORTED_FEATURES['reader.local_cog']` now report `stable`; `reader.http_cog` stays `advanced` while the HTTP transport surface is contracted separately. The stable COG contract covers axis-aligned 2D / 3D rasters, the CPU writer and CPU reader, the lossless codecs (`none`, `deflate`, `lzw`, `zstd`, `packbits`), internal overviews, and normal CRS / transform / dtype / nodata / band / pixel-is-area / pixel-is-point round-trip. GPU COG paths, experimental codecs, rotated transforms, external `.tif.ovr` sidecars, file-like destinations with `cog=True`, BigTIFF COG, and HTTP COG remain outside the contract. Backed by the writer compliance suite (#2292), the cross-backend parity gate (#2293), and the per-tile byte-budget contract (#2294 / #2298). The reference docs (`docs/source/reference/geotiff.rst`) and the COG overview notebook spell out the full contract. (#2300)

README.md

@@ -104,6 +104,9 @@ pip install xarray-spatial
 # with plotting helpers (matplotlib)
 pip install xarray-spatial[plot]
 
+# with vector rasterization (shapely): rasterize, polygonize
+pip install xarray-spatial[vector]
+
 # via conda
 conda install -c conda-forge xarray-spatial
 ```
@@ -617,6 +620,7 @@ Check out the user guide [here](/examples/user_guide/).
 
 **Optional:**
 - `matplotlib` — the `.xrs.plot` accessor helpers (`pip install xarray-spatial[plot]`)
+- `shapely` — the vector-to-raster paths, `rasterize` and `polygonize` (`pip install xarray-spatial[vector]`)
 - `pyproj` — WKT/PROJ CRS resolution
 - `cupy` — GPU acceleration
 - `dask` — out-of-core processing

docs/source/getting_started/installation.rst

@@ -12,8 +12,13 @@ Installation
    # with plotting helpers (matplotlib)
    pip install xarray-spatial[plot]
 
+   # with vector rasterization (shapely): rasterize, polygonize
+   pip install xarray-spatial[vector]
+
    # via conda
    conda install -c conda-forge xarray-spatial
 
-matplotlib is an optional dependency. The compute functions work without it;
-install the ``plot`` extra to use the ``.xrs.plot`` accessor helpers.
+matplotlib and shapely are optional dependencies. The compute functions work
+without either; install the ``plot`` extra for the ``.xrs.plot`` accessor
+helpers, and the ``vector`` extra for the ``rasterize`` and ``polygonize``
+vector-to-raster paths.

setup.cfg

@@ -23,7 +23,6 @@ install_requires =
     scipy
     xarray
     numpy
-    shapely>=2.0
     urllib3
     zstandard
 packages = find:
@@ -57,6 +56,10 @@ plot =
     # import in the package is lazy, so the compute functions work
     # without this extra installed.
     matplotlib
+vector =
+    # Optional for the vector-to-raster paths (rasterize, polygonize).
+    # shapely is imported lazily so `import xrspatial` does not load it.
+    shapely>=2.0
 optional =
     # Optional for polygonize return types.
     awkward>=1.4
@@ -92,6 +95,7 @@ tests =
     pytest
     pytest-cov
     scipy
+    shapely>=2.0
 
 
 [flake8]

xrspatial/rasterize.py

@@ -16,7 +16,6 @@
 from typing import Any, Callable, NamedTuple, Optional, Sequence, Tuple, Union
 
 import numpy as np
-import shapely
 import xarray as xr
 
 from xrspatial.utils import ngjit
@@ -31,6 +30,34 @@
 except ImportError:
     cuspatial = None
 
+#: Cached shapely module, populated by :func:`_require_shapely` on first use.
+#: shapely is an optional dependency (the ``vector`` extra). It is imported
+#: lazily so ``import xrspatial`` does not pull in shapely (and GEOS) for users
+#: who never rasterize vector geometry.
+_shapely = None
+
+
+def _require_shapely():
+    """Import shapely or raise a helpful error, caching the module.
+
+    rasterize and polygonize are the only paths that need shapely. Every
+    function here that touches the shapely array API calls this and binds the
+    return value to a local ``shapely`` name, so the error surfaces clearly
+    (including inside dask workers, which call the tile helpers directly rather
+    than through :func:`rasterize`).
+    """
+    global _shapely
+    if _shapely is None:
+        try:
+            import shapely as _s
+        except ImportError as e:
+            raise ImportError(
+                "shapely is required for rasterize/polygonize but is not "
+                "installed. Install it with: pip install xarray-spatial[vector]"
+            ) from e
+        _shapely = _s
+    return _shapely
+
 
 # ---------------------------------------------------------------------------
 # Allocation guard: reject output dimensions that would exhaust memory
@@ -234,6 +261,7 @@ def _classify_geometries(geometries, props_array):
                 ([], empty_props.copy(), empty_idx.copy()),
                 ([], empty_props.copy(), empty_idx.copy()))
 
+    shapely = _require_shapely()
     type_ids = shapely.get_type_id(geom_arr)
     empty = shapely.is_empty(geom_arr)
     valid = ~empty
@@ -348,6 +376,7 @@ def _extract_edges(geometries, geom_ids, bounds, height, width,
 def _extract_edges_vectorized(geometries, geom_ids, bounds,
                               height, width, all_touched):
     """Vectorized edge extraction using shapely 2.0 array ops."""
+    shapely = _require_shapely()
     xmin, ymin, xmax, ymax = bounds
     px = (xmax - xmin) / width
     py = (ymax - ymin) / height
@@ -472,6 +501,7 @@ def _extract_points(geometries, bounds, height, width):
 
 def _extract_points_vectorized(geometries, bounds, height, width):
     """Vectorized point extraction using shapely 2.0 array ops."""
+    shapely = _require_shapely()
     xmin, ymin, xmax, ymax = bounds
     px = (xmax - xmin) / width
     py = (ymax - ymin) / height
@@ -532,6 +562,7 @@ def _extract_line_segments(geometries, bounds, height, width):
 
 def _extract_lines_vectorized(geometries, bounds, height, width):
     """Vectorized line extraction with Liang-Barsky clipping."""
+    shapely = _require_shapely()
     xmin, ymin, xmax, ymax = bounds
     px = (xmax - xmin) / width
     py = (ymax - ymin) / height
@@ -2131,6 +2162,7 @@ def _geometry_bboxes(geometries):
     """Return (N, 4) float64 array of [xmin, ymin, xmax, ymax] per geometry."""
     if len(geometries) == 0:
         return np.empty((0, 4), dtype=np.float64)
+    shapely = _require_shapely()
     return shapely.bounds(np.asarray(geometries, dtype=object))
 
 
@@ -2298,11 +2330,13 @@ def _polys_to_wkb(geoms):
     """Pre-serialize polygon geometries to WKB for cheap pickling."""
     if not geoms:
         return []
+    shapely = _require_shapely()
     return shapely.to_wkb(np.asarray(geoms, dtype=object)).tolist()
 
 
 def _polys_from_wkb(wkb_list):
     """Deserialize WKB back to shapely geometries."""
+    shapely = _require_shapely()
     geoms = shapely.from_wkb(wkb_list)
     if not isinstance(geoms, (list, np.ndarray)):
         geoms = [geoms]
@@ -3048,6 +3082,10 @@ def rasterize(
         >>> density = rasterize(gdf, width=100, height=100,
         ...                     column='pop', merge='sum', fill=0)
     """
+    # Fail early with a clear message if the optional ``vector`` extra
+    # (shapely) is not installed, rather than deep inside a helper.
+    _require_shapely()
+
     if column is not None and columns is not None:
         raise ValueError(
             "'column' and 'columns' are mutually exclusive; use one or "

xrspatial/tests/test_optional_shapely.py

@@ -0,0 +1,81 @@
+"""Tests for shapely being an optional dependency (the `vector` extra).
+
+shapely backs the vector-to-raster paths (rasterize, polygonize) but is
+imported lazily, so `import xrspatial` works without it. See issue #2496.
+
+These tests run whether or not shapely is installed: they either spawn a
+fresh interpreter with shapely blocked, or block it in ``sys.modules`` and
+reset the cached module in ``xrspatial.rasterize``.
+"""
+import sys
+
+import numpy as np
+import pytest
+import xarray as xr
+
+
+def test_import_xrspatial_without_shapely():
+    """`import xrspatial` and the compute modules work with no shapely.
+
+    Runs in a subprocess so the import happens against a clean module cache
+    with shapely blocked.
+    """
+    import subprocess
+    import textwrap
+
+    code = textwrap.dedent(
+        """
+        import sys
+        sys.modules['shapely'] = None
+
+        import xrspatial  # noqa: F401
+        import xrspatial.focal  # noqa: F401
+        import xrspatial.rasterize  # noqa: F401
+        import xrspatial.polygonize  # noqa: F401
+
+        # shapely must not have been imported as a side effect.
+        if 'shapely' in sys.modules and sys.modules['shapely'] is not None:
+            raise SystemExit('shapely was imported on import xrspatial')
+
+        try:
+            import shapely  # noqa: F401
+        except ImportError:
+            pass
+        else:
+            raise SystemExit('shapely was unexpectedly importable')
+        """
+    )
+    result = subprocess.run(
+        [sys.executable, '-c', code],
+        capture_output=True,
+        text=True,
+    )
+    assert result.returncode == 0, result.stderr
+
+
+def test_require_shapely_message(monkeypatch):
+    """The helper points users at the `vector` extra when shapely is gone."""
+    import importlib
+    rasterize_mod = importlib.import_module('xrspatial.rasterize')
+
+    monkeypatch.setattr(rasterize_mod, '_shapely', None)
+    monkeypatch.setitem(sys.modules, 'shapely', None)
+    with pytest.raises(ImportError, match=r"xarray-spatial\[vector\]"):
+        rasterize_mod._require_shapely()
+
+
+def test_rasterize_without_shapely_raises(monkeypatch):
+    """`rasterize()` raises the friendly error up front when shapely is absent."""
+    import importlib
+    rasterize_mod = importlib.import_module('xrspatial.rasterize')
+
+    monkeypatch.setattr(rasterize_mod, '_shapely', None)
+    monkeypatch.setitem(sys.modules, 'shapely', None)
+
+    template = xr.DataArray(
+        np.zeros((4, 4)),
+        dims=['y', 'x'],
+        coords={'x': np.arange(4.0), 'y': np.arange(4.0)},
+    )
+    with pytest.raises(ImportError, match=r"xarray-spatial\[vector\]"):
+        rasterize_mod.rasterize([], like=template)