docs(proposals): global hook for build-system dependency post-processing

docs/proposals/global-build-system-dependencies-hook.md

@@ -0,0 +1,234 @@
+# Build-system dependency post-processing for cross-cutting concerns
+
+- Author: Vikash Shaw
+- Created: 2026-07-24
+- Status: Proposed
+- GitHub issue: [#1263](https://github.com/python-wheel-build/fromager/issues/1263)
+
+## What
+
+This proposal describes two approaches for handling cross-cutting
+build-system dependency concerns (such as setuptools version capping)
+that affect many packages. Both aim to eliminate the need for identical
+per-package plugins in downstream projects.
+
+## Why
+
+Fromager currently provides two extension mechanisms:
+
+1. **Per-package plugins** (`fromager.project_overrides`): Override a
+   hook for a single package. When present, the plugin replaces the
+   default implementation entirely.
+
+2. **Global hooks** (`fromager.hooks`): Run for every package. Currently
+   support `post_build`, `post_bootstrap`, and `prebuilt_wheel`, which
+   are event callbacks that fire after an action has completed.
+
+Currently, no global hook runs during dependency resolution. When a
+cross-cutting concern affects build dependencies for many packages, the
+only option today is to write identical per-package plugins for each one.
+
+### Motivating example
+
+setuptools 81 removed `distutils.spawn(dry_run=...)` and
+`remove_tree(dry_run=...)`. setuptools 82 removed `pkg_resources`
+entirely. Many PyPI packages still reference these removed APIs in their
+`setup.py`, causing build failures when Fromager resolves an uncapped
+setuptools.
+
+In one downstream project, this led to **22 identical per-package
+plugins**, each scanning `setup.py` to detect removed API usage and
+appending a setuptools version cap. Every time a new package hits the
+same incompatibility, another identical plugin must be added. This does
+not scale well.
+
+### Why not `update_build_requires`?
+
+Fromager's YAML settings support `update_build_requires` for statically
+adding build dependencies. However, the setuptools cap is conditional
+and depends on what APIs a given `setup.py` actually uses. A static YAML
+entry would either over-constrain all packages or require per-package
+entries, which has the same maintenance burden as plugins.
+
+## Proposed approaches
+
+### Option A: Core logic in Fromager (PR [#1264](https://github.com/python-wheel-build/fromager/pull/1264))
+
+Add the setuptools detection logic directly into
+`default_get_build_system_dependencies`. Fromager would automatically
+scan `setup.py` for `pkg_resources` imports and `dry_run` keyword
+arguments, and append the appropriate setuptools version cap to the
+build dependencies.
+
+**Changes in Fromager:**
+
+- Add `_get_setuptools_constraint()` to `dependencies.py` using
+  `ast.parse` to detect removed API usage
+- Call it from `default_get_build_system_dependencies()` and append
+  the constraint if needed
+
+**Changes in downstream projects:**
+
+- None. Delete the 22 identical plugins and their entry points. The
+  packages build correctly without any downstream configuration,
+  plugin, or hook registration.
+
+### Option B: Global hooks (PR [#1271](https://github.com/python-wheel-build/fromager/pull/1271))
+
+Add `get_build_system_dependencies` as a new global hook point under
+`fromager.hooks`. Downstream projects register a hook that post-processes
+the build-system dependencies list for all packages. Fromager stays
+generic and does not include any setuptools-specific logic.
+
+**Changes in Fromager:**
+
+- Add `get_build_system_dependencies` to `GLOBAL_HOOK_NAMES` in
+  `hooks.py`
+- Add `run_get_build_system_dependencies_hooks()` that chains hooks
+  so each receives the previous hook's output
+- Call the hooks in `dependencies.get_build_system_dependencies()`
+  after per-package overrides return, before marker filtering
+
+**Changes in downstream projects:**
+
+- Add a hook implementation (one file with the setuptools detection
+  logic)
+- Register it as an entry point under `fromager.hooks`
+- Delete the 22 identical per-package plugins and their entry points
+- The packages still need to be listed in the downstream project's
+  requirements/collections
+
+Alternatively, the hook could be released as a standalone installable
+package (e.g. `fromager-setuptools-hook`) so any Fromager user can
+opt in by simply installing it, without writing any hook code themselves.
+
+## Comparison
+
+|  | Option A (core) | Option B (global hooks) |
+| -- | -- | -- |
+| Downstream work needed | None | Hook registration or pip install |
+| Fromager stays generic | No, includes setuptools-specific logic | Yes |
+| Other Fromager users benefit | Automatically | Only if they install the hook |
+| Reusable for other concerns | No, only solves setuptools | Yes, hook point is general-purpose |
+| Configurable / opt-out | Would need a config flag | Opt-in by design |
+
+## How (Option A details)
+
+### Execution order
+
+The constraint logic runs inside
+`default_get_build_system_dependencies()`, after reading `[build-system] requires` from `pyproject.toml`:
+
+```
+1. Check for cached requirements file (early return if exists)
+2. overrides.find_and_invoke()          <-- per-package plugin or default
+   2a. default reads [build-system] requires from pyproject.toml
+   2b. _get_setuptools_constraint() scans setup.py   <-- NEW
+   2c. Append constraint if needed
+3. _filter_requirements()               <-- marker evaluation
+4. Write requirements cache file
+```
+
+### Detection logic
+
+`_get_setuptools_constraint(sdist_root_dir)` uses `ast.parse` to walk
+the `setup.py` AST and detect:
+
+- `import pkg_resources` or `from pkg_resources import ...` results in
+  `setuptools<82`
+- `dry_run` keyword argument results in `setuptools<81` (tighter cap
+  takes precedence when both are present)
+- Returns `None` if neither is found, or if `setup.py` does not exist
+- Gracefully returns `None` on `SyntaxError`
+
+### Example
+
+For a package with this `setup.py`:
+
+```python
+from pkg_resources import get_distribution
+from setuptools import setup
+setup(name="example", version="1.0")
+```
+
+Fromager would automatically append `setuptools<82` to its build
+dependencies. No downstream configuration needed.
+
+## How (Option B details)
+
+### Execution order
+
+The hook would run inside `dependencies.get_build_system_dependencies()`,
+after the per-package override (or default) returns and before marker
+filtering:
+
+```
+1. Check for cached requirements file (early return if exists)
+2. overrides.find_and_invoke()          <-- per-package plugin or default
+3. hooks.run_get_build_system_dependencies_hooks()   <-- NEW
+4. _filter_requirements()               <-- marker evaluation
+5. Write requirements cache file
+```
+
+This means per-package plugins still produce the initial dependency
+list, global hooks can then augment it, and marker filtering happens
+last so hooks do not need to handle markers themselves. The result is
+cached, so hooks run only once per package per build.
+
+### Hook signature
+
+```python
+def get_build_system_dependencies(
+    *,
+    ctx: context.WorkContext,
+    req: Requirement,
+    sdist_root_dir: pathlib.Path,
+    build_dir: pathlib.Path,
+    requirements: list[str],
+) -> list[str]:
+    ...
+```
+
+The hook receives the current requirements list and must return a
+(possibly modified) `list[str]`. When multiple hooks are registered,
+they chain: each receives the previous hook's output. Execution order
+follows stevedore's `HookManager` iteration order.
+
+### Registration
+
+Hooks are registered as entry points under the `fromager.hooks`
+namespace, the same way `post_build` and other existing hooks work:
+
+```toml
+[project.entry-points."fromager.hooks"]
+get_build_system_dependencies = "my_package.hooks:get_build_system_dependencies"
+```
+
+### Example hook
+
+A minimal hook that appends a constraint:
+
+```python
+def get_build_system_dependencies(
+    *,
+    ctx: context.WorkContext,
+    req: Requirement,
+    sdist_root_dir: pathlib.Path,
+    build_dir: pathlib.Path,
+    requirements: list[str],
+) -> list[str]:
+    # Inspect sdist content and conditionally add constraints
+    if needs_constraint(build_dir):
+        return requirements + ["setuptools<82"]
+    return requirements
+```
+
+## Interaction with existing mechanisms
+
+| Mechanism | Scope | Relationship |
+| -- | -- | -- |
+| `update_build_requires` (YAML) | Per-package, static | Runs during `prepare_source`, before this hook. |
+| `remove_build_requires` (YAML) | Per-package, static | Same as above. |
+| Per-package plugin | Per-package, dynamic | Runs first. Global hooks receive its output. |
+| Cached `build-system-requirements.txt` | Per-package | If cache exists, function returns early. Hooks do not run. |
+| **Global hooks (Option B)** | All packages, dynamic | Runs after per-package plugin, before marker filtering. |