Skip to content

Conversation

@snazy
Copy link
Member

@snazy snazy commented Nov 7, 2025

This change introduces a new function verify_actions to validate the contents against GitHub.

TL;DR
The function verifies that the SHAs specified in actions.yml exist in the GH repo. Also ensures that the SHA exists on the Git tag, if the tag attribute is specified. The rest of the function is a lot of output and error(failure) and warning collection.

Although it issues quite a few GH API requests, the rate limiter should not kick in (with an authenticated GH token, GH workflows have a limit of 15k requests). I opted to rely on the HTTP/1.1 urllib.request stuff, which has no connection-reuse. The alternative would have been to add a dependency.

The algorithm roughly works like this, for each action specified in actions.yml:

  • Issue a warning and stop, if the name is like OWNER/* ("wildcard" repository). Can't verify Git SHAs in this case.
  • Issue a warning and stop, if the name is like docker:* (not implemented)
  • Issue an error and stop, if the name doesn't start with an OWNER/REPO pattern.
  • Each expired entry is just skipped
  • If there is a wildcard reference and a SHA reference, issue an error.

Then, for each reference for an action:

  • If no tag is specified, let GH resolve the commit SHA. Emit a warning to add the value of the tag attribute, if the SHA can be resolved. Otherwise, emit an error.
  • If tag is specified:
    • Add the SHA to the set of requested-shas-by-tag
    • Call GH's "matching-refs" endpoint for the 'tag' value
      • Emit en error, if the object type is not a tag or commit.
      • Also resolve 'tag' object types to 'commit' object types.
      • Add each returned SHA to the set of valid-shas-by-tag.
  • For each "requested tag" verify that the sets of valid and requested shas intersect. If not, emit an error.

Fixes #110

@snazy
Copy link
Member Author

snazy commented Nov 7, 2025

This is in a very early stage and meant to just gather feedback and opinions about the approach.

@snazy
Copy link
Member Author

snazy commented Nov 17, 2025

@raboof do you think it's worth tackling this one?

@raboof
Copy link
Member

raboof commented Nov 18, 2025

This looks helpful to me. Does it actually share code with gateway.py or could it be a separate file?

@snazy
Copy link
Member Author

snazy commented Nov 18, 2025

This looks helpful to me. Does it actually share code with gateway.py or could it be a separate file?

It does use the load_actions and a data class. But nothing that presents moving the code to a separate .py file.

@snazy snazy force-pushed the gh-action-sha-tag-check branch 19 times, most recently from af91fe6 to 185416b Compare December 21, 2025 14:14
@snazy
Copy link
Member Author

snazy commented Dec 21, 2025

Made some progress on this one.

Moved the code to a separate source file and added it to the update_actions workflow.
The output (github summary) would yield five failures.
One is tackled via #426 (my bad, included in this PR as well for now).

The four remaining ones are because the the ScaCap organization has an IP allow list enabled, which prevents GH hosted runners to perform GH API requests against their org, which prevents the verification code to verify the SHAs and tags. The checks work fine for the ScaCap org from my machine.
I have added a new boolean flag ignore_gh_api_errors for action-references to let the verification code ignore GH API failures. Setting this flag to true means that GH API errors are ignored, but the checks still happen and verification errors are still emitted, just not as failures but as warnings.
Updated the actions.yml with that flag for the scacap/action-surefire-report action.

The warnings are:

  • Two references to Docker images (not verified)
  • Two wildcard repository references
    • golangci/*
    • rustsec/*
  • Two SHAs without a tag name
    • browser-actions/setup-geckodriver
    • damccorm/tag-ur-it
  • Two wildcard SHA and specific SHA references for the same action
    • sbt/setup-sbt
    • gradle/wrapper-validation-action

@snazy snazy force-pushed the gh-action-sha-tag-check branch 6 times, most recently from 6a82461 to a1796d0 Compare December 21, 2025 15:57
@snazy snazy force-pushed the gh-action-sha-tag-check branch 4 times, most recently from 4a17c00 to 41c0e43 Compare December 21, 2025 16:09
@snazy snazy changed the title WIP - verify GH action tag/SHA combinations Verify GH action tag/SHA combinations Dec 21, 2025
@snazy snazy marked this pull request as ready for review December 21, 2025 16:12
This change introduces a new function `verify_actions` to validate the contents against GitHub.

TL;DR
The function verifies that the SHAs specified in `actions.yml` exist in the GH repo.
Also ensures that the SHA exists on the Git tag, if the `tag` attribute is specified.
The rest of the (currently spaghetti code) function is a lot of output and error(failure) and warning collection.

Although it issues quite a few GH API requests, the rate limiter should not kick in (with an authenticated GH token).
I opted to rely on the HTTP/1.1 `urllib.request` stuff, which has no connection-reuse. The alternative would have been to add a dependency.

The algorithm roughly works like this, for each action specified in `actions.yml`:
* Issue a warning and stop, if the name is like `OWNER/*` ("wildcard" repository).
  Can't verify Git SHAs in this case.
* Issue a warning and stop, if the name is like `docker:*` (not implemented)
* Issue an error and stop, if the name doesn't start with an `OWNER/REPO` pattern.
* Each expired entry is just skipped
* If there is a wildcard reference and a SHA reference, issue an error.

Then, for each reference for an action:
* If no `tag` is specified, let GH resolve the commit SHA.
  Emit a warning to add the value of the `tag` attribute, if the SHA can be resolved.
  Otherwise, emit an error.
* If `tag` is specified:
  * Add the SHA to the set of requested-shas-by-tag
  * Call GH's "matching-refs" endpoint for the 'tag' value
    * Emit en error, if the object type is not a tag or commit.
    * Also resolve 'tag' object types to 'commit' object types.
    * Add each returned SHA to the set of valid-shas-by-tag.
* For each "requested tag" verify that the sets of valid and requested shas intersect. If not, emit an error.
@snazy snazy force-pushed the gh-action-sha-tag-check branch from 40c514a to c62528e Compare December 28, 2025 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Verify SHA belongs to released version

2 participants