Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# waybill Development Guidelines

Auto-generated from all feature plans. Last updated: 2026-08-21
Auto-generated from all feature plans. Last updated: 2026-08-23

## Active Technologies
- Rust stable (user-space only; no eBPF touched in this milestone) (002-python-npm-ecosystem)
Expand Down Expand Up @@ -322,6 +322,7 @@ Auto-generated from all feature plans. Last updated: 2026-08-21
- N/A — path-parsing + tiny metadata reads are stateless. (663-cache-probe-resolver)
- Rust stable (workspace toolchain inherited from milestones 001–663; no nightly required for this user-space-only work). + Existing only — `globset = "0.4"` (already a direct workspace dep since milestones 113 + 118; used here for filename-pattern matching), `std::path::{Path, PathBuf}`, `std::fs::{read_dir, canonicalize}`, `std::collections::{HashMap, HashSet}`, `tracing` (INFO-level FR-009 diagnostic log), `anyhow`/`thiserror` (error surface), `serde`/`serde_json` (existing — no new schema). **Zero new Cargo dependencies.** (664-single-pass-walker)
- N/A — the (directory → filenames) index is in-process per scan; dropped at `read_all` return. Mirrors every scan_fs milestone since 002. (664-single-pass-walker)
- Rust stable (workspace toolchain inherited from milestones 001–664; no nightly required) + Existing only — `clap` (workspace; `ValueEnum` derive for the mode enum), `serde`/`serde_json` (annotation values), `tracing` (FR-005 diagnostic log). **Zero new Cargo dependencies.** (665-no-binary-scan-flag)

- Rust stable (user-space) + nightly (eBPF target via `aya-ebpf`) + aya, aya-ebpf, aya-build, tokio, clap, reqwest, serde/serde_json, cyclonedx-bom, packageurl, sha2, chrono, thiserror, anyhow, tracing (001-build-trace-pipeline)

Expand Down Expand Up @@ -415,9 +416,9 @@ of CI-readiness — they are not equivalent.
Rust stable (user-space) + nightly (eBPF target via `aya-ebpf`): Follow standard conventions

## Recent Changes
- 665-no-binary-scan-flag: Added Rust stable (workspace toolchain inherited from milestones 001–664; no nightly required) + Existing only — `clap` (workspace; `ValueEnum` derive for the mode enum), `serde`/`serde_json` (annotation values), `tracing` (FR-005 diagnostic log). **Zero new Cargo dependencies.**
- 664-single-pass-walker: Added Rust stable (workspace toolchain inherited from milestones 001–663; no nightly required for this user-space-only work). + Existing only — `globset = "0.4"` (already a direct workspace dep since milestones 113 + 118; used here for filename-pattern matching), `std::path::{Path, PathBuf}`, `std::fs::{read_dir, canonicalize}`, `std::collections::{HashMap, HashSet}`, `tracing` (INFO-level FR-009 diagnostic log), `anyhow`/`thiserror` (error surface), `serde`/`serde_json` (existing — no new schema). **Zero new Cargo dependencies.**
- 663-cache-probe-resolver: Added Rust stable (workspace toolchain inherited from milestones 001–236; no nightly required). + Existing only —
- 236-unresolved-reason: Added Rust stable (workspace toolchain inherited from milestones 001–235; no nightly required for this user-space-only feature). + Existing only — `serde_json` (annotation value construction; pervasive), `tracing` (existing warn logs on the reader-side design-tier code paths), `anyhow`/`thiserror` (existing error propagation). Reuses the m071 parity extractor infrastructure (`parity/extractors/*.rs`) and the standard `PackageDbEntry.extra_annotations` channel. **Zero new Cargo dependencies.**


<!-- MANUAL ADDITIONS START -->
Expand Down
6 changes: 6 additions & 0 deletions docs/design-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,6 +318,12 @@ Every escape-hatch site carries an inline `// FR-005 permanent escape hatch —
- Rationale for retained walker-audit entries: `waybill-cli/src/scan_fs/walk.audit-allowlist.rationale.md`
- SC-005 microbenchmark: `waybill-cli/tests/perf_walk_dispatch.rs::sc005_synthetic_10k_file_tree_p95_dispatch_overhead`

### Operator-controlled reader gating (milestone 665)

The registration site inside `run_shared_walker_pilot` is also an operator-visible gate. Milestone 665 (`--no-binary-scan=<MODE>` / `WAYBILL_NO_BINARY_SCAN=<MODE>`) uses this seam to elide `go_binary::registration()` at pilot time when the operator opts out of statically-linked-Go BuildInfo probing — trading module attribution for wall-time on large trees (mongo 3.04s → ~0.7s). The gate is a one-line `if !skip_go_binary { ... }` around the existing `register("go_binary", ...)` call; `finalize()` downstream self-elides on the empty candidate-path list, so no separate suppression is needed at the post-pilot site. Setting the flag also emits a document-scope `waybill:binary-scan-suppressed=<mode>` annotation (C153) across CDX / SPDX 2.3 / SPDX 3 so downstream consumers can detect the opt-out without inspecting waybill invocation state. See `specs/665-no-binary-scan-flag/`.

The design intent: `run_shared_walker_pilot` becomes the natural place to gate reader participation by operator preference (or, future, by scan-tier / trust-level policy). The m665 pattern is extensible — the `BinaryScanMode` enum reserves `all` / `elf` / `symbols` variants for future opt-outs of m096 ELF section reader, m099 symbol fingerprint, and m104 binary-role classification without new C-rows.

---

## Key code landmarks
Expand Down
1 change: 1 addition & 0 deletions docs/reference/sbom-format-mapping.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,7 @@ These rows have no native SPDX 2.3 home. Per clarification Q2 (Option A) and FR-
| C142 | `waybill:sbom-version` | document-scope property/annotation — positive integer value carrying the operator-supplied SBOM document version from `waybill sbom scan --sbom-version <N>`. Emitted iff `--sbom-version` was passed on the CLI (silent on the default path, preserving FR-009 byte-identity for pre-m221 goldens). Value type: integer matching CDX 1.6 `metadata.version` schema (`{"type": "integer", "minimum": 1}`). Companion to the native CDX `metadata.version` slot (which always carries the integer, defaulting to `1`): this annotation exposes the SAME value across all three formats for the m071 catalog's cross-format parity check. Milestone 221 (feature 221-cisa-2026-elements-audit, closes the CISA 2026 § SBOM Version minimum-element gap on the SPDX side per FR-013). | Document-level `metadata.properties[]` entry `{"name":"waybill:sbom-version","value":"3"}` (string-encoded integer per CDX 1.6 property-value schema). | Document-level `Annotation` on `SPDXRef-DOCUMENT`, `MikebomAnnotationCommentV1` envelope: `{"schema":"waybill-annotation/v1","field":"waybill:sbom-version","value":3}` (native JSON integer). | Document-scope `Annotation` element on the SpdxDocument root IRI; same envelope shape. | **KEEP-BOTH: native (CDX only) + waybill annotation (all three formats for parity)**. Standards-native audit per Constitution Principle V (bullet 5): CDX 1.6 has native `metadata.version` — waybill populates it from `--sbom-version` when set, defaulting to `1`. The `waybill:sbom-version` annotation is a parity-bridge: it makes the value discoverable via the same annotation-key path across all three emitters, needed by the m071 catalog's cross-format equivalence oracle. SPDX 2.3 has no native SBOM-document-version field distinct from component-version `Package.versionInfo` — the annotation is the primary carrier. SPDX 3 similarly. CISA 2026 § SBOM Version also blesses RFC 9562 UUIDs as an alternative identity pathway; waybill's existing UUID `serialNumber` (CDX) + content-addressed `documentNamespace` / `@id` (SPDX per m010) already satisfy that pathway — this annotation covers the monotonic-integer-counter pathway consumers expect. Value regex: `^[1-9]\d*$`. Milestone 221 US4. |
| C143 | `waybill:pants-resolve` | per-component `properties[]`/annotation — string value carrying the name of the Pants resolve whose Pex lockfile produced the component. Derived from the lockfile filename stem (`3rdparty/python/default.lock` → `"default"`; `3rdparty/python/mypy.lock` → `"mypy"`). Enables downstream auditing of which resolve contributed each Python component. Companion to C42 `waybill:lifecycle-scope` (m179): the pants-pex reader tags components from resolves whose name matches the known-dev-tool allowlist (`mypy`, `pytest`, `black`, `ruff`, etc.) with `Development` scope; C143 preserves the raw resolve name regardless of scope classification. Emitted unconditionally on every pants-pex-derived component (present iff the component came from a Pex lockfile). Milestone 223 US1 (feature 223-pants-pex-reader). | Per-component `properties[]` entry `{"name":"waybill:pants-resolve","value":"mypy"}`. | Per-Package `annotations[].comment` envelope (`MikebomAnnotationCommentV1`): `{"schema":"waybill-annotation/v1","field":"waybill:pants-resolve","value":"mypy"}`. | Per-Package `Annotation.statement` envelope; same shape as SPDX 2.3. | **KEEP-NO-NATIVE**. Standards-native audit per Constitution Principle V (bullet 5): no CDX / SPDX 2.3 / SPDX 3 native carrier for "which build-tool-declared resolve/dep-set this component came from" — the concept is Pants-specific (also relevant to future Bazel / Pants coursier readers). CDX `evidence.identity[].technique` carries the parse-time technique (`manifest-analysis`), not the resolve-name. SPDX 3's `software_Package.additionalPurpose` is a per-component role, not a per-resolve grouping. Annotation preserves the signal losslessly across all three formats for the m071 parity gate. |
| C144 | `waybill:source-url` | per-component `properties[]`/annotation — string value carrying the artifact source URL for components resolved from git URLs, direct HTTP(S) URLs, or local `file://` paths (as opposed to canonical PyPI-hosted wheels/sdists). Emitted **only** on `pkg:generic/*` components produced by the pants-pex reader when the corresponding Pex lockfile entry's `artifacts[0].url` does not start with `https://files.pythonhosted.org/`. Never emitted on `pkg:pypi/*` components (their source-URL is implicit in the PURL type + PyPI's canonical hosting). Composes with C1 `waybill:source-type` (`git` / `url` / `local`) to fully describe the non-PyPI provenance. Milestone 223 US1 + Q2 A clarification (feature 223-pants-pex-reader). | Per-component `properties[]` entry `{"name":"waybill:source-url","value":"git+https://example.test/repo.git@abc123"}`. | Per-Package `annotations[].comment` envelope (`MikebomAnnotationCommentV1`): `{"schema":"waybill-annotation/v1","field":"waybill:source-url","value":"git+https://example.test/repo.git@abc123"}`. | Per-Package `Annotation.statement` envelope; same shape as SPDX 2.3. | **KEEP-NO-NATIVE**. Standards-native audit per Constitution Principle V (bullet 5): CDX `externalReferences[type: vcs]` almost fits git-source entries but conflates "component's canonical VCS repository" (upstream-project metadata) with "specific artifact fetch URL used during resolve" (build-time provenance) — those are distinct concepts (a component's VCS home vs the specific commit/tarball waybill's Pex lockfile pinned). Similarly `externalReferences[type: distribution]` fits direct-URL wheels but not git URLs. SPDX 2.3 `Package.downloadLocation` is closer semantically but is a single-string slot that conflates all source types; the annotation preserves the operator's exact recorded URL without lossy re-encoding. SPDX 3's `software_Package.downloadLocation` has the same shape. **Rejected alternative**: emit into `externalReferences[type: distribution]` OR `Package.downloadLocation` — rejected because the shape conflates git / direct-URL / local-file semantics (see C1 for the source-type discriminator). |
| C153 | `waybill:binary-scan-suppressed` | document-scope `properties[]`/annotation — string value naming which binary-scanning reader(s) the operator opted out of via `--no-binary-scan=<MODE>` (or the `WAYBILL_NO_BINARY_SCAN` env-var equivalent). Emitted **iff and only if** the flag was set — the default (unset) path is byte-identical to pre-m665 emission per FR-003. Closed enum (v1 recognizes only `go`, matching the `#[clap(name)]` attribute on `BinaryScanMode::Go`). Value MUST match `BinaryScanMode::as_annotation_value()` at `waybill-cli/src/cli/scan_cmd.rs` (single source of truth). Future releases may add `all` / `elf` / `symbols` variants without new C-rows — this annotation's value slot is designed for enum extension. Enables downstream consumers (vuln scanners, compliance auditors) to detect that the SBOM excludes go_binary-derived components without inspecting waybill invocation state. Milestone 665 (feature 665-no-binary-scan-flag). | doc-scope `metadata.properties[]` entry `{"name":"waybill:binary-scan-suppressed","value":"go"}`. | Document-scope `annotations[].comment` envelope (`MikebomAnnotationCommentV1`): `{"schema":"waybill-annotation/v1","field":"waybill:binary-scan-suppressed","value":"go"}`. | Document-scope `Annotation.statement` envelope; same shape as SPDX 2.3. | **KEEP-NO-NATIVE**. Standards-native audit per Constitution Principle V: no CDX / SPDX 2.3 / SPDX 3 native carrier for "which binary-content reader(s) the operator opted out of". CDX `metadata.tools[].note` is human-readable freeform, not machine-actionable. SPDX `creationInfo.creators` names the tool but not its per-invocation opt-outs. SPDX 3 `CreationInfo.createdUsing` mirrors CDX's tools field. Annotation preserves the transparency signal (Constitution Principle X) losslessly across all three formats. Similar shape + rationale to C146 (`waybill:gradle-resolution-tier`) — both are doc-scope closed-enum annotations naming which subsystem produced (or didn't produce) data. |
| C152 | `waybill:resolver-tier` | per-component `properties[]`/annotation — string value naming which resolver tier produced this component's identity in the attestation-consumer resolution pipeline. Closed enum matching the wire form of `ResolutionTechnique` (`url_pattern` / `local_cache_hit` / `hash_match` / `package_database` / `file_path_pattern` / `hostname_heuristic`). **MVP scope (m663)**: emitted only on components produced by the cache-probe resolver (value `local_cache_hit`). Universal emission across all resolvers is planned as a follow-on. Milestone 663 (feature 663-cache-probe-resolver, closes issue #605). | Per-component `properties[]` entry `{"name":"waybill:resolver-tier","value":"local_cache_hit"}`. | Per-Package `annotations[].comment` envelope (`MikebomAnnotationCommentV1`): `{"schema":"waybill-annotation/v1","field":"waybill:resolver-tier","value":"local_cache_hit"}`. | Per-Package `Annotation.statement` envelope; same shape as SPDX 2.3. | **KEEP-NO-NATIVE**. Standards-native audit per Constitution Principle V: no CDX / SPDX 2.3 / SPDX 3 native carrier for "which resolver tier produced this component's identity." CDX `evidence.identity[].technique` names the parse-time technique from a fixed CDX vocabulary (`manifest-analysis`, `filename`, `hash-comparison`, etc.); the waybill resolver tier is a finer-grained internal distinction. SPDX 3 `evidence.identity[]` mirrors CDX. Annotation preserves the operator-actionable signal losslessly across all three formats. Similar shape + rationale to C147 `waybill:gradle-fallback-reason` — both name a specific resolution mechanism. |
| C151 | `waybill:unresolved-reason` | per-component `properties[]`/annotation — string value naming the reader-specific resolution boundary that produced this design-tier component. Emitted iff and only if the same component carries `waybill:sbom-tier: "design"`. Value examples: `"no matching entry in Podfile.lock"` (cocoapods), `"no <version> in pom.xml; no dependency-reduced-pom.xml or effective-pom fallback"` (maven), `"unrendered Chart.yaml dependency; --helm-render subprocess disabled or unavailable"` (helm). Full per-reader enumeration locked in `specs/236-unresolved-reason/contracts/per-reader-strings.md`. Universalization of the NuGet-only precedent established by PR #656 across all 13 design-tier-emitting readers (cocoapods, composer, dart, elixir, erlang, gradle_static, haskell, helm, kotlin_dsl/build_script, maven, npm/walk, nuget, pants_go, pants_shell, pip, scala, yocto). Per Q1 clarification, reason strings are **display-only** — byte-stable within a waybill build, best-effort stable across releases; downstream tools MUST render verbatim and MUST NOT parse for programmatic branching. Milestone 236 (feature 236-unresolved-reason, closes issue #659). | Per-component `properties[]` entry `{"name":"waybill:unresolved-reason","value":"no matching entry in Podfile.lock"}`. | Per-Package `annotations[].comment` envelope (`MikebomAnnotationCommentV1`): `{"schema":"waybill-annotation/v1","field":"waybill:unresolved-reason","value":"no matching entry in Podfile.lock"}`. | Per-Package `Annotation.statement` envelope; same shape as SPDX 2.3. | **KEEP-NO-NATIVE**. Standards-native audit per Constitution Principle V: no CDX / SPDX 2.3 / SPDX 3 native carrier for "human-readable name of the resolution boundary that produced this specific unresolved component". CDX `evidence.identity[].technique` names the parse-time technique (`manifest-analysis`), not the specific resolver-failure reason. CDX `evidence.identity[].confidence` is numeric — not a remediation string. SPDX 3 `LifecycleScope` describes runtime scope, not resolver failure. SPDX `Annotation.comment` is generic — waybill's `MikebomAnnotationCommentV1` envelope IS the vehicle. Annotation preserves the operator-actionable signal losslessly across all three formats. Similar shape + rationale to C109 `waybill:go-transitive-unresolved-reason` — both name a specific resolution boundary. |
| C150 | `waybill:gradle-platform-import` | per-component `properties[]`/annotation — string value carrying the sorted, comma-joined `g:a:v` list of Gradle `platform(...)` and `enforcedPlatform(...)` BOM imports declared in the enclosing project's `build.gradle`. Emitted **only** on components produced by the US3 static-parser tier when the enclosing project declares at least one BOM import; absent on components from US1/US2/lockfile-only tiers and from projects without BOM imports (byte-identity preserved). BOM imports themselves do NOT emit as components (they're version-constraint declarations, not artifacts) — this annotation is how operators trace which BOMs govern the emitted dep set. Example: `com.example.waybillfixture:bom-parent:1.0.0` (single BOM) or `com.example.waybillfixture:bom-a:1.0.0,com.example.waybillfixture:bom-b:2.0.0` (multiple, lex-sorted). Companion to C148 (per-component tier) and C149 (per-component cache-freshness); all three ride on the standard `PackageDbEntry.extra_annotations` channel. Milestone 235 US4 follow-on (feature 235-gradle-transitive-ladder). | Per-component `properties[]` entry `{"name":"waybill:gradle-platform-import","value":"com.example.waybillfixture:bom-parent:1.0.0"}`. | Per-Package `annotations[].comment` envelope (`MikebomAnnotationCommentV1`): `{"schema":"waybill-annotation/v1","field":"waybill:gradle-platform-import","value":"com.example.waybillfixture:bom-parent:1.0.0"}`. | Per-Package `Annotation.statement` envelope; same shape as SPDX 2.3. | **KEEP-NO-NATIVE**. Standards-native audit per Constitution Principle V: no CDX / SPDX 2.3 / SPDX 3 native carrier for "which BOM version-constraint declarations govern the dep set for the emitted component's containing project." CDX `metadata.component.dependencies[]` describes runtime dep relationships between concrete components, not BOM version-constraint declarations. SPDX 3 `Package.suppliedBy` describes upstream supplier, not BOM version-governance. `pkg:maven/...?type=pom` PURL variant is closest but conflates BOM POMs with regular POM-packaging artifacts. Annotation preserves the signal losslessly across all three formats. |
Expand Down
Loading
Loading