Skip to content

Commit d4dffa2

Browse files
msarahanmsarahan
authored
Add release build output dispatch action (#134)
# What is this? This is the reusable action that creates manifests for the artifacts produced by our open source builds. This is part of a plan for: * rapidsai/release-scripts#102 * [For scanning purposes, especially of binary artifacts like wheels, we need to know what software was used at build time. It is often not possible to reverse engineer this information with what we have today.](https://gitlab-master.nvidia.com/RAPIDS/nspect-manager/-/merge_requests/5) The implementation here follows our existing dispatch pattern, rather than earlier efforts that basically did the same thing on shared-workflows (rapidsai/shared-workflows#609)
1 parent c530ab5 commit d4dffa2

11 files changed

Lines changed: 1079 additions & 0 deletions

File tree

‎.github/workflows/pr.yml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,3 +20,34 @@ jobs:
2020
fetch-depth: 1
2121
persist-credentials: false
2222
- uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
23+
- name: Test release build-output materialization
24+
run: |
25+
./tests/release_build_output_descriptors_test.sh
26+
./tests/release_build_output_prepare_test.sh
27+
./tests/release_build_output_test.sh
28+
- name: Prepare release build-output dispatch smoke test
29+
run: |
30+
mkdir -p release-build-output-smoke
31+
printf '%s\n' smoke >release-build-output-smoke/package.tar.gz
32+
- name: Run release build-output dispatch smoke test
33+
uses: ./release-build-output-dispatch
34+
env:
35+
SHARED_ACTIONS_REPO: ${{ github.event.pull_request.head.repo.full_name }}
36+
SHARED_ACTIONS_REF: ${{ github.event.pull_request.head.sha }}
37+
with:
38+
artifact-type: custom
39+
output-directory: release-build-output-smoke
40+
release-artifacts: '[{"path":"package.tar.gz"}]'
41+
release-package: '{"ecosystem":"archive","name":"smoke","version":"1.0"}'
42+
release-unit: archive:smoke
43+
source-artifact-name: release-build-output-dispatch-smoke
44+
source-sha: ${{ github.event.pull_request.head.sha }}
45+
- name: Verify release build-output dispatch smoke test
46+
run: |
47+
jq -e '
48+
.artifacts[0].unit_id == "archive:smoke"
49+
and .artifacts[0].path == "package.tar.gz"
50+
' release-build-output-smoke/release-build-output.json >/dev/null
51+
jq -e '
52+
.metadata.artifacts == [{path: "package.tar.gz", sbom_kind: "generated-identity"}]
53+
' release-build-output-smoke/release-build-metadata.json >/dev/null

‎README.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,41 @@ A dispatch action is one that:
99
* clones the shared-actions repository (repo/ref changeable using env vars)
1010
* runs (dispatches to) another action within the clone, using a relative path
1111

12+
## Release build-output companions
13+
14+
`release-build-output-dispatch` validates a producer's local build artifact
15+
directory and uploads a companion artifact named
16+
`release-build-output-<source-artifact-name>`. The companion contains
17+
`release-build-output.json`, `release-build-metadata.json`, provenance, and an
18+
SBOM record for every primary artifact.
19+
20+
Conda and wheel jobs can set `artifact-type` to `conda` or `wheel` and omit
21+
`release-artifacts`; the implementation reads package metadata from the built
22+
files. Custom bundles provide explicit artifact descriptors and either inline
23+
package identity or a producer-created package JSON file.
24+
25+
```yaml
26+
- name: Create release build-output companion
27+
uses: rapidsai/shared-actions/release-build-output-dispatch@main
28+
with:
29+
artifact-type: wheel
30+
output-directory: ${{ steps.package-name.outputs.WHEEL_OUTPUT_DIR }}
31+
release-unit: wheel:example
32+
source-artifact-name: ${{ steps.package-name.outputs.RAPIDS_PACKAGE_NAME }}
33+
source-sha: ${{ github.sha }}
34+
```
35+
36+
A descriptor-selected producer SBOM is classified as `producer-dependency`.
37+
When no SBOM is supplied, the action generates an SPDX artifact-identity
38+
envelope and classifies it as `generated-identity`. The generated envelope
39+
contains the primary artifact's identity and SHA-256 but no dependency
40+
inventory; it must not be treated as dependency coverage.
41+
42+
The dispatch wrapper honors `SHARED_ACTIONS_REPO` and `SHARED_ACTIONS_REF`.
43+
When neither is set, it checks out the same repository and ref used to invoke
44+
the wrapper, which allows a feature-branch wrapper to dispatch to its matching
45+
implementation during canary testing.
46+
1247
There can be more complicated arrangements of more actions, but the idea is to
1348
have the local clone of the shared-actions repository be the first step of an action.
1449

‎release-build-output-dispatch/action.yml‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
name: Dispatch release build output
2+
description: Check out the selected shared-actions revision and create a release build-output companion.
3+
4+
inputs:
5+
artifact-type:
6+
description: One of conda, wheel, or custom.
7+
required: true
8+
release-unit:
9+
description: Release-platform unit ID for every primary artifact in this bundle.
10+
required: true
11+
release-package:
12+
description: JSON package fields shared by the bundle.
13+
required: false
14+
release-package-file:
15+
description: Relative path to producer-created package JSON inside output-directory.
16+
required: false
17+
release-artifacts:
18+
description: JSON artifact and evidence descriptors relative to output-directory.
19+
required: false
20+
output-directory:
21+
description: Directory containing the primary files and any producer-supplied evidence.
22+
required: true
23+
manifest-name:
24+
description: Filename to write inside output-directory.
25+
required: false
26+
default: release-build-output.json
27+
metadata-name:
28+
description: Filename for the build metadata envelope.
29+
required: false
30+
default: release-build-metadata.json
31+
source-artifact-name:
32+
description: Name of the GitHub Actions artifact bundle containing this output.
33+
required: true
34+
source-sha:
35+
description: Source revision built by the producing job.
36+
required: false
37+
38+
outputs:
39+
manifest-path:
40+
description: Absolute path to the generated manifest.
41+
value: ${{ steps.release-build-output.outputs.manifest-path }}
42+
metadata-path:
43+
description: Absolute path to the build metadata envelope.
44+
value: ${{ steps.release-build-output.outputs.metadata-path }}
45+
manifest-artifact-name:
46+
description: Name of the uploaded GitHub Actions companion artifact.
47+
value: ${{ steps.release-build-output.outputs.manifest-artifact-name }}
48+
49+
runs:
50+
using: composite
51+
steps:
52+
- name: Check out shared-actions implementation
53+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
54+
with:
55+
repository: ${{ env.SHARED_ACTIONS_REPO || github.action_repository || 'rapidsai/shared-actions' }}
56+
ref: ${{ env.SHARED_ACTIONS_REF || github.action_ref || 'main' }}
57+
path: ./shared-actions
58+
persist-credentials: false
59+
- id: release-build-output
60+
name: Create release build-output companion
61+
uses: ./shared-actions/release-build-output
62+
with:
63+
artifact-type: ${{ inputs.artifact-type }}
64+
release-unit: ${{ inputs.release-unit }}
65+
release-package: ${{ inputs.release-package }}
66+
release-package-file: ${{ inputs.release-package-file }}
67+
release-artifacts: ${{ inputs.release-artifacts }}
68+
output-directory: ${{ inputs.output-directory }}
69+
manifest-name: ${{ inputs.manifest-name }}
70+
metadata-name: ${{ inputs.metadata-name }}
71+
source-artifact-name: ${{ inputs.source-artifact-name }}
72+
source-sha: ${{ inputs.source-sha }}

‎release-build-output/action.yml‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
name: Create release build output companion
2+
description: Validate a build artifact bundle and upload its release-platform manifest and evidence companion.
3+
4+
inputs:
5+
artifact-type:
6+
description: One of conda, wheel, or custom. Conda and wheel descriptors are derived when release-artifacts is omitted.
7+
required: true
8+
release-unit:
9+
description: Release-platform unit ID for every primary artifact in this bundle.
10+
required: true
11+
release-package:
12+
description: JSON package fields shared by the bundle. Provide this or release-package-file for custom artifacts.
13+
required: false
14+
release-package-file:
15+
description: Relative path to producer-created package JSON inside output-directory.
16+
required: false
17+
release-artifacts:
18+
description: JSON artifact and evidence descriptors relative to output-directory. Required for custom artifacts.
19+
required: false
20+
output-directory:
21+
description: Directory containing the primary files and any producer-supplied evidence.
22+
required: true
23+
manifest-name:
24+
description: Filename to write inside output-directory.
25+
required: false
26+
default: release-build-output.json
27+
metadata-name:
28+
description: Filename for the build-environment and SBOM-classification envelope.
29+
required: false
30+
default: release-build-metadata.json
31+
source-artifact-name:
32+
description: Name of the GitHub Actions artifact bundle containing this output.
33+
required: true
34+
source-sha:
35+
description: Source revision built by the producing job. Defaults to the current workflow SHA.
36+
required: false
37+
38+
outputs:
39+
manifest-path:
40+
description: Absolute path to the generated manifest.
41+
value: ${{ steps.materialize.outputs.manifest-path }}
42+
metadata-path:
43+
description: Absolute path to the build metadata envelope.
44+
value: ${{ steps.materialize.outputs.metadata-path }}
45+
manifest-artifact-name:
46+
description: Name of the uploaded GitHub Actions companion artifact.
47+
value: ${{ steps.companion-name.outputs.name }}
48+
49+
runs:
50+
using: composite
51+
steps:
52+
- id: prepare
53+
name: Describe release artifacts
54+
shell: bash
55+
env:
56+
RELEASE_ARTIFACTS: ${{ inputs.release-artifacts }}
57+
RELEASE_ARTIFACT_TYPE: ${{ inputs.artifact-type }}
58+
RELEASE_OUTPUT_DIRECTORY: ${{ inputs.output-directory }}
59+
RELEASE_PACKAGE: ${{ inputs.release-package }}
60+
RELEASE_PACKAGE_FILE: ${{ inputs.release-package-file }}
61+
run: ./shared-actions/release-build-output/prepare.sh
62+
- id: materialize
63+
name: Materialize release build-output records
64+
shell: bash
65+
env:
66+
RELEASE_ARTIFACTS: ${{ steps.prepare.outputs.artifacts }}
67+
RELEASE_MANIFEST_NAME: ${{ inputs.manifest-name }}
68+
RELEASE_METADATA_NAME: ${{ inputs.metadata-name }}
69+
RELEASE_OUTPUT_DIRECTORY: ${{ inputs.output-directory }}
70+
RELEASE_PACKAGE: ${{ steps.prepare.outputs.package }}
71+
RELEASE_PACKAGE_FILE: ${{ inputs.release-package-file }}
72+
RELEASE_SOURCE_ARTIFACT_NAME: ${{ inputs.source-artifact-name }}
73+
RELEASE_SOURCE_SHA: ${{ inputs.source-sha || github.sha }}
74+
RELEASE_UNIT: ${{ inputs.release-unit }}
75+
run: ./shared-actions/release-build-output/materialize.sh
76+
- id: companion-name
77+
name: Set companion artifact name
78+
shell: bash
79+
env:
80+
SOURCE_ARTIFACT_NAME: ${{ inputs.source-artifact-name }}
81+
run: echo "name=release-build-output-${SOURCE_ARTIFACT_NAME}" >>"${GITHUB_OUTPUT}"
82+
- name: Upload release build-output companion
83+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
84+
with:
85+
if-no-files-found: error
86+
name: ${{ steps.companion-name.outputs.name }}
87+
path: |
88+
${{ inputs.output-directory }}/${{ inputs.manifest-name }}
89+
${{ inputs.output-directory }}/${{ inputs.metadata-name }}
90+
${{ inputs.output-directory }}/release-evidence/**

‎release-build-output/describe-conda.sh‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
#!/usr/bin/env bash
2+
# Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
4+
set -euo pipefail
5+
6+
if [[ "$#" -ne 1 || ! -d "$1" ]]; then
7+
echo "usage: $0 CONDA_OUTPUT_DIRECTORY" >&2
8+
exit 1
9+
fi
10+
11+
output_directory="$(realpath "$1")"
12+
descriptors='[]'
13+
package_count=0
14+
15+
while IFS= read -r package_path; do
16+
package_path="$(realpath "${package_path}")"
17+
if [[ "${package_path}" != "${output_directory}"/* ]]; then
18+
echo "Conda package must resolve inside output directory: ${package_path}" >&2
19+
exit 1
20+
fi
21+
relative_path="${package_path#"${output_directory}/"}"
22+
23+
case "${package_path}" in
24+
*.conda)
25+
info_members=()
26+
while IFS= read -r info_member; do
27+
info_members+=("${info_member}")
28+
done < <(unzip -Z1 "${package_path}" | awk '/^info-.*\.tar\.zst$/')
29+
if [[ "${#info_members[@]}" -ne 1 ]]; then
30+
echo ".conda package must contain exactly one info-*.tar.zst member: ${relative_path}" >&2
31+
exit 1
32+
fi
33+
index_json="$(unzip -p "${package_path}" "${info_members[0]}" | zstd -dc | tar -xOf - info/index.json)"
34+
;;
35+
*.tar.bz2)
36+
index_json="$(tar -xOjf "${package_path}" info/index.json)"
37+
;;
38+
*)
39+
echo "unsupported Conda package extension: ${relative_path}" >&2
40+
exit 1
41+
;;
42+
esac
43+
44+
if ! jq -e '
45+
type == "object"
46+
and (.name | type == "string" and length > 0)
47+
and (.version | type == "string" and length > 0)
48+
and (.build | type == "string" and length > 0)
49+
and (.subdir | type == "string" and length > 0)
50+
' <<<"${index_json}" >/dev/null; then
51+
echo "Conda info/index.json must contain exact name, version, build, and subdir fields: ${relative_path}" >&2
52+
exit 1
53+
fi
54+
55+
package="$(jq -c '{ecosystem: "conda", name, version, build, platform: .subdir}' <<<"${index_json}")"
56+
descriptors="$(jq -cn \
57+
--arg path "${relative_path}" \
58+
--argjson package "${package}" \
59+
--argjson current "${descriptors}" \
60+
'$current + [{path: $path, package: $package}]')"
61+
package_count=$((package_count + 1))
62+
done < <(find "${output_directory}" -type f \( -name '*.conda' -o -name '*.tar.bz2' \) -print | sort)
63+
64+
if [[ "${package_count}" -eq 0 ]]; then
65+
echo "Conda output directory contains no .conda or .tar.bz2 packages: ${output_directory}" >&2
66+
exit 1
67+
fi
68+
69+
printf '%s\n' "${descriptors}"

‎release-build-output/describe-wheels.sh‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
#!/usr/bin/env bash
2+
# Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
4+
set -euo pipefail
5+
6+
if [[ "$#" -ne 1 || ! -d "$1" ]]; then
7+
echo "usage: $0 WHEEL_OUTPUT_DIRECTORY" >&2
8+
exit 1
9+
fi
10+
11+
output_directory="$(realpath "$1")"
12+
descriptors='[]'
13+
wheel_count=0
14+
15+
while IFS= read -r wheel_path; do
16+
wheel_path="$(realpath "${wheel_path}")"
17+
if [[ "${wheel_path}" != "${output_directory}"/* ]]; then
18+
echo "wheel must resolve inside output directory: ${wheel_path}" >&2
19+
exit 1
20+
fi
21+
relative_path="${wheel_path#"${output_directory}/"}"
22+
23+
metadata_members=()
24+
while IFS= read -r metadata_member; do
25+
metadata_members+=("${metadata_member}")
26+
done < <(unzip -Z1 "${wheel_path}" | awk '/\.dist-info\/METADATA$/')
27+
if [[ "${#metadata_members[@]}" -ne 1 ]]; then
28+
echo "wheel must contain exactly one .dist-info/METADATA file: ${relative_path}" >&2
29+
exit 1
30+
fi
31+
32+
metadata="$(unzip -p "${wheel_path}" "${metadata_members[0]}")"
33+
package_name="$(awk 'tolower($0) ~ /^name:[[:space:]]*/ {sub(/^[^:]*:[[:space:]]*/, ""); sub(/\r$/, ""); print; exit}' <<<"${metadata}")"
34+
package_version="$(awk 'tolower($0) ~ /^version:[[:space:]]*/ {sub(/^[^:]*:[[:space:]]*/, ""); sub(/\r$/, ""); print; exit}' <<<"${metadata}")"
35+
if [[ -z "${package_name}" || -z "${package_version}" ]]; then
36+
echo "wheel metadata must contain non-empty Name and Version fields: ${relative_path}" >&2
37+
exit 1
38+
fi
39+
40+
descriptors="$(jq -cn \
41+
--arg path "${relative_path}" \
42+
--arg name "${package_name}" \
43+
--arg version "${package_version}" \
44+
--argjson current "${descriptors}" \
45+
'$current + [{path: $path, package: {ecosystem: "wheel", name: $name, version: $version}}]')"
46+
wheel_count=$((wheel_count + 1))
47+
done < <(find "${output_directory}" -type f -name '*.whl' -print | sort)
48+
49+
if [[ "${wheel_count}" -eq 0 ]]; then
50+
echo "wheel output directory contains no .whl files: ${output_directory}" >&2
51+
exit 1
52+
fi
53+
54+
printf '%s\n' "${descriptors}"

0 commit comments

Comments
 (0)