diff --git a/site/content/community/release-guide.md b/site/content/community/release-guide.md index 90d4ef68a0..488d58e39e 100644 --- a/site/content/community/release-guide.md +++ b/site/content/community/release-guide.md @@ -17,7 +17,7 @@ # specific language governing permissions and limitations # under the License. # -linkTitle: Release Guide +linkTitle: Release Manager Guide type: docs weight: 500 --- diff --git a/site/content/community/release-verify.md b/site/content/community/release-verify.md new file mode 100644 index 0000000000..adc615e768 --- /dev/null +++ b/site/content/community/release-verify.md @@ -0,0 +1,25 @@ +--- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +linkTitle: Release Verification Guide +type: docs +weight: 500 +--- + +{{< readfile "/release-verify.md" >}} diff --git a/site/content/release-guide.md b/site/content/release-guide.md index 45a64dbf96..fc78de6ae0 100644 --- a/site/content/release-guide.md +++ b/site/content/release-guide.md @@ -17,9 +17,13 @@ under the License. --> -# Release Guide +# Release Manager Guide -This guide walks you through the release process of the Apache Polaris podling. +**Audience**: Release Managers + +This guide walks you through the process of **creating** a release of the Apache Polaris podling. + +Instructions how to verify a release candidate are available [here](release-verify.md). ## Setup diff --git a/site/content/release-verify.md b/site/content/release-verify.md new file mode 100644 index 0000000000..62c5b9b99a --- /dev/null +++ b/site/content/release-verify.md @@ -0,0 +1,252 @@ + + +# Release Verification Guide + +**Audience**: Committers and interested contributors. + +This guide walks you through the process of **verifying** a staged Apache Polaris release candidate. + +Verifying a (staged) release of an Apache project has to follow a bunch of tasks, which can be +grouped into tasks that can be automated and those that need human intervention. +Polaris provides a tool to automate the tasks that can be automated. + +Tasks that are automated: +* Checksums and PGP signatures are valid. +* All expected artifacts are present. +* Source code artifacts have correct names matching the current release. +* Built artifacts are [reproducible](#reproducible-builds). +* Build passes. +* `DISCLAIMER`, `LICENSE` and `NOTICE` files are included. +* main and sources jar artifacts contain `META-INF/LICENSE` and `META-INF/NOTICE` files. +* main distribution artifacts contain `DISCLAIMER`, `LICENSE` and `NOTICE` files in the top-level directory. + +Tasks that need human intervention: +* Download links are valid. Check all links in the `[VOTE]` email for the release: + * Tag on the GitHub website + * Commit on the GitHub website + * SVN repository with the source tarball and binary release artifacts + * SVN repository with the Helm chart + * Link to the KEYS file (_MUST_ be equal to `https://downloads.apache.org/incubator/polaris/KEYS`) + * Maven staging repository +* `DISCLAIMER`, `LICENSE` and `NOTICE` files are correct for the repository. +* Contents of jar artifacts `META-INF/LICENSE` and `META-INF/NOTICE` files are correct. +* All files have license headers if necessary. + This is (mostly) verified using the "rat" tool during builds/CI. +* No disallowed binary artifacts are bundled in the source archive. + This is a (soft) requirement to be held true by committers. + +**Imply good intent!** +Although the release manager is responsible for producing a "proper" release, mistakes can and will happen. +The Polaris project is committed to providing reproducible builds as an essential building block of +_Apache trusted releases_. +The project depends on frameworks which also strive to provide reproducible builds, but not all +these frameworks can provide fully reproducible builds yet. +The Polaris project's release verification tool will therefore report some issues that are currently expected. +See [below](#reproducible-builds) for details. + +# Verifying a release candidate + +Instead of performing all mentioned steps manually, you can leverage the script +`tools/verify-release/verify-release.sh` available in the main repository to perform the +automatable tasks. + +Always run the most recent version of the script using the following command: +```bash +bash <(curl \ + -s https://raw.githubusercontent.com/apache/polaris/refs/heads/main/tools/verify-release/verify-release.sh) \ + --help +``` + +The tool is intended for Polaris versions 1.3 and newer. +The tool may report issues, see [below](#reproducible-builds) for details. + +That script requires a couple of tools installed and will check that those are available +and report those that need to be installed. + +To run the script, you need the following pieces of information: +* The *full* Git SHA of the corresponding source commit. +* The version number of the release, something like `1.3.0` +* The RC number of the release, for example `1` or `2` +* The Maven staging repository ID, for example `1033` (the full number at the end of the Maven repository URL `https://repository.apache.org/content/repositories/orgapachepolaris-1033/`). + +Example (values taken from the 1.2.0-rc2 release) +```bash +bash <(curl \ + -s https://raw.githubusercontent.com/apache/polaris/refs/heads/main/tools/verify-release/verify-release.sh) \ + --git-sha 354a5ef6b337bf690b7a12fefe2c984e2139b029 \ + --version 1.2.0 \ + --rc 2 \ + --maven-repo-id 1033 +``` + +Same example, but using the short option names: +```bash +bash <(curl \ + -s https://raw.githubusercontent.com/apache/polaris/refs/heads/main/tools/verify-release/verify-release.sh) \ + -s 354a5ef6b337bf690b7a12fefe2c984e2139b029 \ + -v 1.2.0 \ + -r 2 \ + -m 1033 +``` + +The verification script creates a temporary directory that will eventually contain a fresh Git clone, +all downloaded and all built artifacts. +This temporary directory is deleted after the script has finished. To keep the temporary directory +around, you can use the `--keep-temp-dir` (`-k`) option. + +A log file, the name matches the pattern `polaris-release-verify-*.log`, +will be created and contain detailed information about the identified issues reported on the console. + +Note: The script is maintained in the Polaris source tree in the `/tools/verify-release` directory. + +## Verifications performed by the tool + +After some startup checks, the tool emits some information about the release candidate. For example: +``` +Verifying staged release +======================== + +Git tag: apache-polaris-1.2.0-incubating-rc2 +Git sha: 354a5ef6b337bf690b7a12fefe2c984e2139b029 +Full version: 1.2.0-incubating +Maven repo URL: https://repository.apache.org/content/repositories/orgapachepolaris-1033/ +Main dist URL: https://dist.apache.org/repos/dist/dev/incubator/polaris/1.2.0-incubating +Helm chart URL: https://dist.apache.org/repos/dist/dev/incubator/polaris/helm-chart/1.2.0-incubating +Verify directory: /tmp/polaris-release-verify-2025-10-23-14-22-31-HPmmiybzk + +A verbose log containing the identified issues will be available here: + /home/snazy/devel/polaris/polaris/polaris-release-verify-2025-10-23-14-22-31.log +``` + +After that, release candidate verification starts immediately, performing the following operations: + +1. Create `gpg` keyring for signature verification from the project's `KEYS` file. +2. Clone the Git repository directly from GitHub. +3. Git commit SHA and Git tag match +4. Verifies that the mandatory files are present in the source tree +5. Helm chart GPG signature and checksum checks +6. Source tarball and binary artifacts GPG signature and checksum checks +7. Build Polaris from the Git commit/tag +8. Compares that the list of Maven artifacts produced by the build matches those in the Nexus staging repository. +9. Compares the individual Maven artifacts of the local build with the ones in the Nexus staging repository. +10. Compares the main binary distribution artifacts for Polaris server and Polaris admin tool. +11. Compares the locally built Helm chart with the one in the staging repository. + +Found issues are reported on the console in _red_. + +Details for each reported issue will be reported in the log file, depending on the issue and file type. +The intent is to provide as much information as possible to eventually fix a reproducible build issue. +* Text files: Output of `diff` of the local and staged files. +* Zip/Jar files: output of `zipcmp`. + If `zipcmp` reports no difference, the output of `zipinfo` for both files is logged. +* Tarballs: output of `diff --recursive` the extracted local and staged tarballs. + If `diff` reports no difference, the output of `tar tvf` for both files is logged. +* Other files: Output of `diff` of the local and staged files. + +Note: GPG signatures are verified **only** against the project's `KEYS` file. + +# Reproducible builds + +A build is reproducible if the built artifacts are identical on every build from the same source. + +The Apache Polaris build is currently mostly reproducible, with some release-version specific exceptions. + +## Exceptions for all Apache Polaris versions + +Pending on full support for reproducible builds in Quarkus: +* Jars containing generated code are not guaranteed to be reproducible. Affects the following jars: + * */quarkus/generated-bytecode.jar + * */quarkus/transformed-bytecode.jar + * */quarkus/quarkus-application.jar +* Re-assembled jars are not guaranteed to be reproducible: Affects the following jars: + * admin/app/polaris-admin-*.jar + * server/app/polaris-server-*.jar +* Zips and tarballs containing any of the above are not guaranteed to be reproducible. + +Helm chart package tarball is not binary reproducible because there is no option to influence the +mtime and POSIX attributes of the archive entries. +The actual content of the archive entries is reproducible. + +## Exceptions for Apache Polaris up to 1.2 (including) + +* Depending on the operating system being used by the release manager and the "verifier," jar and zip files + might be reported as different, even if the content of the jar and zip files is identical. + This also leads to reported differences of the Gradle *.module files, because the checksums are different. + Fixed via https://github.com/apache/polaris/pull/2819 +* Source tarball is not binary reproducible because of non-constant mtime for tar entries. + Fixed via https://github.com/apache/polaris/pull/2823 +* The content of the parent pom contains dynamically generated content for the lists of developers and + contributors. + Fixed via https://github.com/apache/polaris/pull/2826 + +The following reported issues are expected, depending on the user's environment (`umask` likely). +``` +-------------------------------------------------------------------------- + Comparing Maven repository artifacts, this will take a little while... +-------------------------------------------------------------------------- +Locally built and staged Maven repository artifact org/apache/polaris/polaris-admin/1.2.0-incubating/polaris-admin-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-api-catalog-service/1.2.0-incubating/polaris-api-catalog-service-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-api-iceberg-service/1.2.0-incubating/polaris-api-iceberg-service-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-api-management-model/1.2.0-incubating/polaris-api-management-model-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-api-management-service/1.2.0-incubating/polaris-api-management-service-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-async-api/1.2.0-incubating/polaris-async-api-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-async-java/1.2.0-incubating/polaris-async-java-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-async-vertx/1.2.0-incubating/polaris-async-vertx-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-config-docs-annotations/1.2.0-incubating/polaris-config-docs-annotations-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-config-docs-generator/1.2.0-incubating/polaris-config-docs-generator-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-container-spec-helper/1.2.0-incubating/polaris-container-spec-helper-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-core/1.2.0-incubating/polaris-core-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-eclipselink/1.2.0-incubating/polaris-eclipselink-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-extensions-federation-hadoop/1.2.0-incubating/polaris-extensions-federation-hadoop-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-extensions-federation-hive/1.2.0-incubating/polaris-extensions-federation-hive-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-idgen-api/1.2.0-incubating/polaris-idgen-api-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-idgen-impl/1.2.0-incubating/polaris-idgen-impl-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-idgen-mocks/1.2.0-incubating/polaris-idgen-mocks-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-idgen-spi/1.2.0-incubating/polaris-idgen-spi-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-immutables/1.2.0-incubating/polaris-immutables-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-minio-testcontainer/1.2.0-incubating/polaris-minio-testcontainer-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-misc-types/1.2.0-incubating/polaris-misc-types-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-persistence-nosql-varint/1.2.0-incubating/polaris-persistence-nosql-varint-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-relational-jdbc/1.2.0-incubating/polaris-relational-jdbc-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-runtime-common/1.2.0-incubating/polaris-runtime-common-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-runtime-defaults/1.2.0-incubating/polaris-runtime-defaults-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-runtime-service/1.2.0-incubating/polaris-runtime-service-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-runtime-spark-tests/1.2.0-incubating/polaris-runtime-spark-tests-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-runtime-test-common/1.2.0-incubating/polaris-runtime-test-common-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-server/1.2.0-incubating/polaris-server-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-spark-3.5_2.12/1.2.0-incubating/polaris-spark-3.5_2.12-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-spark-3.5_2.13/1.2.0-incubating/polaris-spark-3.5_2.13-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-spark-integration-3.5_2.12/1.2.0-incubating/polaris-spark-integration-3.5_2.12-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-spark-integration-3.5_2.13/1.2.0-incubating/polaris-spark-integration-3.5_2.13-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-tests/1.2.0-incubating/polaris-tests-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris-version/1.2.0-incubating/polaris-version-1.2.0-incubating.module differ +Locally built and staged Maven repository artifact org/apache/polaris/polaris/1.2.0-incubating/polaris-1.2.0-incubating.pom differ + + +----------------------------------------- + Comparing main distribution artifacts +----------------------------------------- +Locally built and staged Polaris distribution tarball polaris-bin-1.2.0-incubating.tgz differ +Locally built and staged Polaris distribution zip polaris-bin-1.2.0-incubating.zip differ +``` + +## Exceptions for Apache Polaris up to 1.1 (including) + +Apache Polaris builds up to 1.1 are not reproducible. diff --git a/site/hugo.yaml b/site/hugo.yaml index 33116ebcf5..f357ccfed7 100644 --- a/site/hugo.yaml +++ b/site/hugo.yaml @@ -157,10 +157,14 @@ menu: parent: "community" url: "/community/security-report" weight: 70 - - name: "Release Guide" + - name: "Release Manager Guide" parent: "community" url: "/community/release-guide" weight: 80 + - name: "Release Verification Guide" + parent: "community" + url: "/community/release-verify" + weight: 81 - name: "Blogs" parent: "community" url: "/blog" diff --git a/tools/verify-release/README.md b/tools/verify-release/README.md new file mode 100644 index 0000000000..e544a5fdbc --- /dev/null +++ b/tools/verify-release/README.md @@ -0,0 +1,22 @@ + + +Helpers and information for verifying a release candidate can be found +on the Polaris website: https://polaris.apache.org/ or +in the Polaris source tree in the `site/content/release-verify.md` file. diff --git a/tools/verify-release/verify-release.sh b/tools/verify-release/verify-release.sh new file mode 100755 index 0000000000..cfc26a9576 --- /dev/null +++ b/tools/verify-release/verify-release.sh @@ -0,0 +1,531 @@ +#!/usr/bin/env bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +set -e + +maven_repo_url_prefix="https://repository.apache.org/content/repositories/orgapachepolaris-" + +function usage() { + cat << ! > /dev/stderr + +Apache Polaris release candidate verification tool. + +Usage: $0 [options] + + Mandatory options: + -s | --git-sha | --sha Git commit (full, not abbreviated) + Example: b7188a07511935e7c9c64128dc047107c26f97f6 + -v | --version Release version (without RC and 'incubating') + Example: 1.2.0 + -r | --rc RC number (without a leading 'rc') + Example: 1 + -m | --maven-repo-id Staging Maven repository staging ID + Example: 1032 + This will be prefixed with ${maven_repo_url_prefix} + + Optional arguments: + -k | --keep-temp-dir Keep the temporary directory (default is to purge it once the script exits) + -h | --help Show usage information (exits early) + + +Full example for RC1 of 1.2.0, staging repo ID 1032. + ./verify-release.sh -s b7188a07511935e7c9c64128dc047107c26f97f6 -v 1.2.0 -r 1 -m 1032 +! +} + +git_sha="" +version="" +rc_num="" +maven_repo_id="" +keep_temp_dir=0 + +while [[ $# -gt 0 ]]; do + arg="$1" + case "$arg" in + -s | --git-sha | --sha) + git_sha="$2" + shift + ;; + -v | --version) + version="$2" + shift + ;; + -r | --rc) + rc_num="$2" + shift + ;; + -m | --maven-repo-id) + maven_repo_id="$2" + shift + ;; + -h | --help) + usage + exit 0 + ;; + -k | --keep-temp-dir) + keep_temp_dir=1 + ;; + esac + shift +done + +RED='\033[0;31m' +ORANGE='\033[0;33m' +RESET='\033[m' + +run_id="polaris-release-verify-$(date "+%Y-%m-%d-%k-%M-%S")" +temp_dir="$(mktemp --tmpdir --directory "${run_id}-XXXXXXXXX")" +function purge_temp_dir { + if [[ $keep_temp_dir -eq 0 ]] ; then + echo "Purging ${temp_dir}..." + rm -rf "${temp_dir}" + else + echo "Leaving ${temp_dir} around, you may want to purge it." + fi +} +trap purge_temp_dir EXIT + +dist_dir="${temp_dir}/dist" +helm_dir="${temp_dir}/helm" +helm_work_dir="${temp_dir}/helm_work" +worktree_dir="${temp_dir}/worktree" +maven_repo_dir="${temp_dir}/maven-repo" +maven_local_dir="${temp_dir}/maven-local" +keys_file="${temp_dir}/KEYS" +gpg_keyring="${temp_dir}/keyring.gpg" + +failures_file="$(pwd)/${run_id}.log" + +dist_url_prefix="https://dist.apache.org/repos/dist/dev/incubator/polaris/" +keys_file_url="https://downloads.apache.org/incubator/polaris/KEYS" + +version_full="${version}-incubating" +git_tag_full="apache-polaris-${version_full}-rc${rc_num}" + +GITHUB=0 +[[ -n ${GITHUB_ENV} ]] && GITHUB=1 + +# Common excludes for "find' +find_excludes=( + # Exclude GPG signatures and checksums + '!' '-name' '*.asc' + '!' '-name' '*.md5' + '!' '-name' '*.sha1' + '!' '-name' '*.sha256' + '!' '-name' '*.sha512' + # file with that name is created by wget when mirroring from 'dist' + '!' '-name' "${version_full}" + # ignore Maven repository metadata + '!' '-name' 'maven-metadata*.xml' + '!' '-name' 'archetype-catalog.xml' +) + +dist_url="${dist_url_prefix}${version_full}" +helm_url="${dist_url_prefix}helm-chart/${version_full}" +maven_repo_url="${maven_repo_url_prefix}${maven_repo_id}/" + +function log_part_start { + local heading + local separator + heading="${*}" + [ ${GITHUB} == 1 ] && echo "::group::$heading" + echo "" + # shellcheck disable=SC2046 + separator="--$(printf -- '-%0.s' $(eval "echo {1..${#heading}}"))--" + echo "${separator}" + echo " ${heading}" + echo "${separator}" +} + +function log_part_end { + [ ${GITHUB} == 1 ] && echo "::endgroup::" + echo "" +} + +function log_fatal { + echo -n -e "${RED}" + echo -n "$1" + echo -e "${RESET}" + echo "" >> "${failures_file}" + for i in "${@}"; do + echo "$i" >> "${failures_file}" + done +} + +function log_warn { + echo -ne "${ORANGE}" + echo -n "$1" + echo -e "${RESET}" +} + +function log_info { + echo "$1" +} + +# Executes a process and captures a fatal error if the process did not complete successfully. +# The full output of the process execution will be logged in the FAILURES file, but not printed to the console. +# First argument: log message for 'log_fatal' +# Following arguments: process arguments +function proc_exec { + local err_msg + local output + err_msg=$1 + shift + output=() + IFS=$'\n' read -r -d '' -a output < <( "${@}" 2>&1 && printf '\0' ) || ( + log_fatal "${err_msg}" "${output[@]}" + return 1 + ) +} + +function mirror { + local url + local dir + local cut + local wget_executable + url="$1" + dir="$2" + cut="$3" + wget_executable="wget" + # Prefer wget2 as it allows parallel downloads (wget does not) + (which wget2 > /dev/null) && wget_executable="wget2 --max-threads=8" + log_part_start "Mirroring $url, this may take a while..." + mkdir -p "${dir}" + (cd "${dir}" ; ${wget_executable} \ + --no-parent \ + --no-verbose \ + --no-host-directories \ + --mirror \ + -e robots=off \ + --cut-dirs="${cut}" \ + "${url}/") + # Nuke the directory listings (index.html from server) and robots.txt... + # (only wget2 downloads the robots.txt :( ) + find "${dir}" \( -name index.html -o -name robots.txt \) -exec rm {} + + find "${dir}" -name "*.prov" | while read -r helmProv; do + if gunzip -c "${helmProv}" > /dev/null 2>&1 ; then + mv "${helmProv}" "${helmProv}.gz" + gunzip "${helmProv}.gz" + fi + done || log_fatal "find failed, please try again" + log_part_end +} + +function verify_checksums { + local dir + dir="$1" + log_part_start "Verifying signatures and checksums in ${dir} ..." + find "${dir}" -mindepth 1 -type f "${find_excludes[@]}" | while read -r fn ; do + echo -n ".. $fn ... " + if [[ -f "$fn.asc" ]] ; then + echo -n "sig " + proc_exec "$fn : Invalid signature" gpg --no-default-keyring --keyring "${gpg_keyring}" --verify "$fn.asc" "$fn" || true + else + log_fatal "$fn : Mandatory ASC signature missing" + fi + if [[ -f "$fn.sha512" ]] ; then + echo -n "sha512 " + provided="$(cut -d\ -f1 < "$fn.sha512")" || log_fatal "sha512 provided $fn failed" + calc="$(shasum -a 512 "$fn" | cut -d\ -f1)" || log_fatal "sha512 calc $fn failed" + [[ "$provided" != "$calc" ]] && log_fatal "$fn : Expected SHA512 $calc - provided $provided" + else + log_fatal "$fn : Mandatory SHA512 missing" + fi + if [[ -f "$fn.sha256" ]] ; then + echo -n "sha256 " + provided="$(cut -d\ -f1 < "$fn.sha256")" || log_fatal "sha256 provided $fn failed" + calc="$(shasum -a 256 "$fn" | cut -d\ -f1)" || log_fatal "sha256 calc $fn failed" + [[ "$provided" != "$calc" ]] && log_fatal "$fn : Expected SHA256 $calc - provided $provided" + fi + if [[ -f "$fn.sha1" ]] ; then + echo -n "sha1 " + provided="$(cut -d\ -f1 < "$fn.sha1")" || log_fatal "sha1 provided $fn failed" + calc="$(shasum -a 1 "$fn" | cut -d\ -f1)" || log_fatal "sha1 calc $fn failed" + [[ "$provided" != "$calc" ]] && log_fatal "$fn : Expected SHA1 $calc - provided $provided" + fi + if [[ -f "$fn.md5" ]] ; then + echo -n "md5 " + provided="$(cut -d\ -f1 < "$fn.md5")" || log_fatal "md5 provided $fn failed" + calc="$(md5sum "$fn" | cut -d\ -f1)" || log_fatal "md5 calc $fn failed" + [[ "$provided" != "$calc" ]] && log_fatal "$fn : Expected MD5 $calc - provided $provided" + fi + echo "" + done || log_fatal "find failed, please try again" + log_part_end +} + +function report_mismatch { + local local_file + local repo_file + local title + local_file="$1" + repo_file="$2" + title="$3" + case "${local_file}" in + *.jar | *.zip) + if proc_exec "${title}" zipcmp "${local_file}" "${repo_file}"; then + # Dump ZIP info only when the contents are equal (to inspect mtime and posix attributes) + ( + log_warn "${title}" + echo ">>>>>>>>>>>>>>> zipinfo ${local_file}" + zipinfo "${local_file}" || true + echo ">>>>>>>>>>>>>>> zipinfo ${repo_file}" + zipinfo "${repo_file}" || true + echo "" + ) >> "${failures_file}" + fi + ;; + *.tar.gz | *.tgz | *.tar) + mkdir "${local_file}.extracted" "${repo_file}.extracted" + tar --warning=no-timestamp -xf "${local_file}" --directory "${local_file}.extracted" || true + tar --warning=no-timestamp -xf "${repo_file}" --directory "${repo_file}.extracted" || true + if proc_exec "${title}" diff --recursive "${local_file}.extracted" "${repo_file}.extracted"; then + # Dump tar listing only when the contents are equal (to inspect mtime and posix attributes) + log_warn "${title}" + ( + echo "${title}" # log_warn above prints ANSI escape sequences + echo ">>>>>>>>>>>>>>> tar tvf ${local_file}" + tar --warning=no-timestamp -tvf "${local_file}" || true + echo ">>>>>>>>>>>>>>> tar tvf ${repo_file}" + tar --warning=no-timestamp -tvf "${repo_file}" || true + echo "" + ) >> "${failures_file}" + fi + ;; + *) + log_fatal "${title}" + ( + diff "${local_file}" "${repo_file}" || true + echo "" + ) >> "${failures_file}" + ;; + esac +} + +function compare_binary_file { + local name + local filename + local local_file + local repo_file + name="$1" + filename="$2" + local_file="$3/${filename}" + repo_file="$4/${filename}" + if ! diff "${local_file}" "${repo_file}" > /dev/null ; then + report_mismatch "${local_file}" "${repo_file}" "Locally built and staged $name $filename differ" + fi +} + +missing_tools=() +for mandatory_tool in wget gunzip find git helm java gpg md5sum shasum tar curl zipcmp zipinfo ; do + if ! which "${mandatory_tool}" > /dev/null; then + missing_tools+=("${mandatory_tool}") + fi +done +if [[ ${#missing_tools} -ne 0 ]]; then + log_fatal "Mandatory tools ${missing_tools[*]} are missing, please install those first." + exit 1 +fi +if ! which wget2 > /dev/null; then + log_warn "For improved website mirroring performance install 'wget2' as it allows multi-threaded downloads." +fi + +if [[ -z $git_sha || -z $version || -z $rc_num || -z $maven_repo_id ]]; then + echo "Mandatory parameter missing" > /dev/stderr + usage + exit 1 +fi + +touch "${failures_file}" + +cat << ! + +Verifying staged release +======================== + +Git tag: ${git_tag_full} +Git sha: ${git_sha} +Full version: ${version_full} +Maven repo URL: ${maven_repo_url} +Main dist URL: ${dist_url} +Helm chart URL: ${helm_url} +Verify directory: ${temp_dir} + +A verbose log containing the identified issues will be available here: + ${failures_file} + +! + +log_part_start "Create verification keyring from ${keys_file_url} ..." +curl --silent --output "${keys_file}" "${keys_file_url}" +gpg --no-default-keyring --keyring "${gpg_keyring}" --import "${keys_file}" +log_part_end + +# Git dance: +# Fetch from remotes and create a local Git worktree for the RC tag, verify Git SHAs. +log_part_start "Git checkout tag ${git_tag_full}, expecting ${git_sha}" +mkdir -p "${worktree_dir}" +(cd "${worktree_dir}" + proc_exec "git init failed" git init . 2> /dev/null + proc_exec "git config failed" git config --local gc.auto 0 + proc_exec "git remote add failed " git remote add origin https://github.com/apache/polaris.git + proc_exec "git fetch failed" git fetch origin tag "${git_tag_full}" + proc_exec "git checkout failed" git checkout "${git_tag_full}" +) +git_sha_on_tag="$(cd "${worktree_dir}" ; git rev-parse HEAD)" +git_sha_from_tag="$(cd "${worktree_dir}" ; git rev-parse "${git_tag_full}")" +log_info "Git commit from tag '${git_tag_full}': ${git_sha_from_tag}" +log_info "Git commit on tag '${git_tag_full}': ${git_sha_on_tag}" +if [[ "$git_sha_on_tag" != "$git_sha_from_tag" ]]; then + log_fatal "Git SHA ${git_sha_from_tag} on ${git_tag_full} is different from the current SHA ${git_sha_on_tag}" +fi +if [[ "$git_sha_on_tag" != "$git_sha" ]]; then + log_fatal "Expected Git SHA ${git_sha} is different from the current SHA ${git_sha_on_tag}" +fi +log_part_end + +log_part_start "Verify mandatory files in source tree" + [[ -e "${worktree_dir}/DISCLAIMER" ]] || log_fatal "Mandatory DISCLAIMER file missing in source tree" + [[ -e "${worktree_dir}/LICENSE" ]] || log_fatal "Mandatory LICENSE file missing in source tree" + [[ -e "${worktree_dir}/NOTICE" ]] || log_fatal "Mandatory NOTICE file missing in source tree" + [[ "$(cat "${worktree_dir}/version.txt")" == "${version_full}" ]] || log_fatal "version.txt in source tree does not contain expected version" +log_part_end + +# Mirror the helm chart content for the release, verify signatures and checksums +mirror "${helm_url}" "${helm_dir}" 7 +verify_checksums "${helm_dir}" +# Mirror the main distribution content for the release, verify signatures and checksums +mirror "${dist_url}" "${dist_dir}" 6 +verify_checksums "${dist_dir}" +# Mirror the repository.apache.org content for the release, verify signatures and checksums +mirror "${maven_repo_url}" "${maven_repo_dir}" 3 +verify_checksums "${maven_repo_dir}" + +# Build Polaris ("assemble") and publish to a separate local Maven repository +log_part_start "Building Polaris ..." +mkdir -p "${maven_local_dir}" +(cd "${worktree_dir}" ; ./gradlew \ + -Dmaven.repo.local="${maven_local_dir}" \ + publishToMavenLocal \ + sourceTarball \ + assemble \ + -PjarWithGitInfo +) +log_part_end + +# Check that the the set of locally built Maven artifacts and staged Maven artifacts is the same. +log_part_start "Comparing Maven build artifacts ..." +find "${maven_local_dir}" -mindepth 2 -type f "${find_excludes[@]}" -printf '%P\n' \ + | sort \ + > "${temp_dir}/maven-local-files" +find "${maven_repo_dir}" -mindepth 2 -type f "${find_excludes[@]}" -printf '%P\n' \ + | sort \ + > "${temp_dir}/maven-repo-files" +proc_exec "List of locally build Maven artifacts and staged artifacts differs!" \ + diff "${temp_dir}/maven-local-files" "${temp_dir}/maven-repo-files" || true +log_part_end + +# Verify that the locally built Maven artifacts are reproducible (binary equal) +log_part_start "Comparing Maven repository artifacts, this will take a little while..." +while read -r fn ; do + compare_binary_file "Maven repository artifact" "${fn}" "${maven_local_dir}" "${maven_repo_dir}" + # verify that the "main" and sources jars contain LICENSE + NOTICE files + [[ "${fn}" =~ .*-$version_full(-sources)?[.]jar ]] && ( + if [[ $(zipinfo -1 "${maven_repo_dir}/${fn}" | grep --extended-regexp --count "^META-INF/(LICENSE|NOTICE)$") -ne 2 ]] ; then + log_fatal "${fn}: Mandatory LICENSE/NOTICE files not in META-INF/" + fi + ) +done < "${temp_dir}/maven-local-files" +log_part_end + +log_part_start "Comparing main distribution artifacts" +compare_binary_file "source tarball" "apache-polaris-${version_full}.tar.gz" "${worktree_dir}/build/distributions" "${dist_dir}" +dist_file_prefix="polaris-bin-${version_full}" +compare_binary_file "Polaris distribution tarball" "${dist_file_prefix}.tgz" "${worktree_dir}/runtime/distribution/build/distributions" "${dist_dir}" +if [[ $(tar -tf "${dist_dir}/${dist_file_prefix}.tgz" | grep --extended-regexp --count "^${dist_file_prefix}/(DISCLAIMER|LICENSE|NOTICE)$") -ne 3 ]] ; then + log_fatal "${dist_file_prefix}.tgz: Mandatory DISCLAIMER/LICENSE/NOTICE files not in ${dist_file_prefix}/" +fi +compare_binary_file "Polaris distribution zip" "${dist_file_prefix}.zip" "${worktree_dir}/runtime/distribution/build/distributions" "${dist_dir}" +if [[ $(zipinfo -1 "${dist_dir}/${dist_file_prefix}.zip" | grep --extended-regexp --count "^${dist_file_prefix}/(DISCLAIMER|LICENSE|NOTICE)$") -ne 3 ]] ; then + log_fatal "${dist_file_prefix}.zip: Mandatory DISCLAIMER/LICENSE/NOTICE files not in ${dist_file_prefix}/" +fi +log_part_end + +log_part_start "Comparing helm chart artifacts" +mkdir -p "${helm_work_dir}/local" "${helm_work_dir}/staged" +proc_exec "Helm packaging failed" helm package --destination "${helm_work_dir}" "${worktree_dir}/helm/polaris" +helm_package_file="polaris-${version_full}.tgz" +tar --warning=no-timestamp -xf "${helm_dir}/${helm_package_file}" --directory "${helm_work_dir}/staged" || true +tar --warning=no-timestamp -xf "${helm_work_dir}/${helm_package_file}" --directory "${helm_work_dir}/local" || true +proc_exec "Helm package ${helm_package_file} contents" diff -r "${helm_work_dir}/local" "${helm_work_dir}/staged" +[[ -e "${helm_work_dir}/staged/polaris/DISCLAIMER" ]] || log_fatal "Mandatory DISCLAIMER file missing in Helm package ${helm_package_file}" +[[ -e "${helm_work_dir}/staged/polaris/LICENSE" ]] || log_fatal "Mandatory LICENSE file missing in Helm package ${helm_package_file}" +[[ -e "${helm_work_dir}/staged/polaris/NOTICE" ]] || log_fatal "Mandatory NOTICE file missing in Helm package ${helm_package_file}" +log_part_end + +if [[ -s ${failures_file} ]] ; then + cat << ! + +************************************************************************************************************ +** Automatic release check FAILED ! +************************************************************************************************************ + +One or more staged release artifacts did not pass the required checks. +A detailed report is available in the file + + ${failures_file} + +INSPECT THE CONTENTS OF THE ABOVE FILE _BEFORE_ REPORTING THE RELEASE CONTENTS AS INVALID! + +* Git SHA mismatches MUST be treated as fatal. +* GPG signature verification errors MUST be treated as fatal. +* Checksum mismatches MUST be treated as fatal. +* Files being reported as missing MUST be treated as fatal. + +The Polaris build is not yet fully reproducible. +A list of known reproducible build issues is maintained in https://github.com/apache/polaris/issues/2204. + +Pending on full support for reproducible builds in Quarkus: +* Jars containing generated code are not guaranteed to be reproducible. Affects the following jars: + * */quarkus/generated-bytecode.jar + * */quarkus/transformed-bytecode.jar + * */quarkus/quarkus-application.jar +* Re-assembled jars are not guaranteed to be reproducible: Affects the following jars: + * admin/app/polaris-admin-*.jar + * server/app/polaris-server-*.jar +* Zips and tarballs containing any of the above are not guaranteed to be reproducible. + +! + exit 1 +else + cat << ! +************************************************************************************************************ +** Automatic release check succeeded +************************************************************************************************************ + +None of the implemented automatic staged release checks reported a mismatch or failure. +* The source tarball matches the contents at the referenced Git commit. +* GPG signatures and checksums are valid and correct. +* The locally built release artifacts are binary equal to the staged release artifacts. + +The contents of all LICENSE and NOTICE files however MUST be verified manually. + +! +fi