signing_tools: macOS Signing and Notarization Tools

This set of scripts currently provides macOS signing and notarization tools for command-line binaries.

The macOS signing and notarization tools (macos_sign.sh and macos_notarize.sh) must be run on macOS.

Examples:

./macos_sign.sh --signing-password="${SIGNING_TOOLS_SIGNING_PASSWORD}" --cert-file=${CERTFILE} --cert-name="${CERTNAME}" --target-binary="${TARGET_BINARY}"

./macos_notarize.sh --app-specific-password=${APP_SPECIFIC_PASSWORD} --apple-id=${APPLE_ID} --primary-bundle-id=com.ddev.test-signing-tools --target-binary=${TARGET_BINARY} [ --team-id=<short-id> ]

The rest of this file explains the methods and resources for signing.

macOS Command-line Binary Signing and Notarization

DDEV and other tools use this to do macOS signing and notarization.

Overview

Apple's ongoing initiatives at controlling what runs on their platforms took a new turn with macOS Catalina (10.15), with required app and command-line binary signing.

Notarization requires

• An Apple Developer Program organization membership from developer.apple.com
• Obtaining a signing cert from Apple.
• Signing the binary or app with a Developer ID Certificate (not a distribution cert)
• Notarization (uploading the binary to Apple for approval)
• Validating code signing
• Validating notarization
• An app-specific password created on your Apple account.
• You may need a new "Developer Relations Intermediate Certificate". From https://developer.apple.com/forums/thread/662300 :

  Just download the certificate from here and install it. If it doesn't works have a look on https://developer.apple.com/support/expiration/

Creating and exporting the signing certificate

• Signing requires the one-time task of obtaining a doing a certificate request (and creating associated private key) and downloading the certificate. See docs.
  • Open Keychain Access and go to Certificate Assistant -> Request a Certificate from a Certificate Authority
  • Provide the User Email Address and a Common Name identifier and save to disk.
  • Sign in with organization owner credentials at developer.apple.com
  • At "Certificates, Identiifiers & Profiles" click the + to create a new certificate.
  • Choose "Developer ID Application"
  • Upload the CSR you created.
  • Download the created certificate (it's a `.cer file).
  • Open the downloaded cert in Keychain Access
  • In "My Certificates" export the cert at a .p12 file (it absolutely must be a .p12 file)
  • Export the new cert with a password and store it as a 1Password attachment; do not commit the .p12 or private key to Git.

Signing a command-line binary

• The process requires that binaries be hardened and signed with the Developer ID certificate, so, for example, DDEV's Apple account on developer.apple.com might have a cert called 'Developer ID Application: DDEV Foundation (9HQ298V2BW)'. This cert can be used for signing multiple binaries or applications.
• Signing is done with the macOS tool codesign. For example, codesign --keychain buildagent -s 'Developer ID Application: DDEV Foundation (9HQ298V2BW)' --timestamp --options runtime .gotmp/bin/darwin_amd64/ddev. The macos_sign.sh tool here just codifies that process.
• G2-issued identities require the Developer ID G2 intermediate certificate when signing. macos_sign.sh downloads Apple's public intermediate into a temporary file and imports it into the temporary signing keychain.

Validating the signature on the binary

Signature validation can be done with codesign -v, for example, codesign -vv -d .gotmp/bin/darwin_amd64/ddev.

Notarizing a binary

Notarizing a binary means

• Uploading the signed binary to Apple for its approval
• Verifying that the process completes successfully and has no warnings
• Verifying from the build process (a link given at notarization completed) that there are no warnings. (When I first got notarization to work, it reported that the package was accepted, but there was a warning that it did not have a "Developer ID" certificate, and thus was not successful.)
• In the case of a .app or other types of artifact, "stapling" the approval to the artifact. In the case of a command-line binary it is not possible to staple the approval. Apple announcement specifies that stapling is for apps, installer packages, and kernel extensions. We can expect this to be added in the future for command-line binaries, but at this time there is no place in the binary architecture for anything to be stapled. The Apple notarizing article says

  Although tickets are created for standalone binaries, it’s not currently possible to staple tickets to them.

Validating notarization

The best technique I've found for validating succesful notarization was archichect, which validates the signing and also checks in with Apple to see if it's been notarized.

codesign --test-requirement="=notarized" --verify --verbose ddev was suggested as an approach, but it doesn't seem to work on a binary that can't be stapled.

CI-based Signing and Notarization

Signing and Notarizing are implemented in DDEV's Makefile and make darwin_signed there does the whole process using the tools from this repo.

This repository's real signing and notarization test runs only in the protected GitHub Actions signing environment. It retrieves the shared DDEV Developer ID identity from 1Password at job runtime and removes the temporary .p12 before the job exits. See 1PASSWORD_SETUP.md for the required vault item and service-account setup.

The job runs automatically for same-repository pull requests, but must be approved through the protected signing environment before it can access credentials. Fork pull requests run only PR-safe validation.

Running the signing integration test locally

Maintainers with access to the required 1Password vaults can run the real test without retaining a local certificate export:

brew install bats-core bats-core/bats-core/bats-assert bats-core/bats-core/bats-file bats-core/bats-core/bats-support
APPLE_ID='the DDEV notarization Apple ID' make signing-integration-test

The command requires an authenticated op CLI session and access to ddev-signing and test-secrets. It reads the certificate and credentials only for the test process, and removes the temporary certificate afterward. The CERTIFICATE_REF, CERTIFICATE_PASSWORD_REF, and APP_SPECIFIC_PASSWORD_REF environment variables may override the default 1Password references when testing a rotated identity. APPLE_ID is not secret; use the same address configured as the GitHub Actions variable.

Resources and Links

• Apple regularly changes their developer agreements. Every time they do, you have to agree to the change before notarizing will work. You have to sign into your apple account and then visit appstoreconnect.apple.com to accept the agreement. (When trying to notarize, you'll get "Error: Unable to notarize app." and "Error: code 1048 (You must first sign the relevant contracts online. (1048))" from altool.)
• Basic Step-by-step Signing and Notarization Walkthrough
• Testing Notarization and Archichect validation tool
• Notarization Answer on Stack Overflow
• notarize-app script for CI notarization.
• Apple's general Notarizing article
• Apple's Customizing the Notarization Workflow article
• CI Signing technique (Stack Overflow)
• Signing without the popup password (for CI, same SO question)
• Apple's Code Signing docs
• Common Code Signing Errors

Developer and Contribution information

• If you're making changes, use make signing-integration-test to test them (see Running the signing integration test locally above). It requires an authenticated op CLI session and access to the ddev-signing and test-secrets 1Password vaults, since the tests need a real Developer ID certificate (CERTFILE/CERTNAME/TEAM_ID) to sign against. bats tests / make test will fail with unset-variable errors without these, since there is no checked-in test certificate to fall back to.
• Forked PRs will not run tests in this repo, because they could expose the APP_SPECIFIC_PASSWORD.