Summary cards are not configurable

[szczys]

We are using alls-green for a dozen or so verify steps in our workflow. This results in many different summary cards which for the most part (reporting successful matrix steps) obscure more useful information. This behavior should be configurable. I plan to open a pull request to address this.

[webknjaz]

Interesting.. It was never really meant to be called multiple times, though. Maybe start with documenting the use case for that here?

[szczys]

Our SDK uses Github Actions to run tests on all support platforms and hardware. Here's a general outline:

PR or Merge
|  |  |
|  |  device-testing matrix
|  |        |
|  |        |--- device 1 --> sample test matrix --> alls-green verifies 12-20 tests in matrix
|  |        |
|  |        |--- device 2 --> sample test matrix --> alls-green verifies 12-20 tests in matrix
|  |        |
|  |        |-- ...
|  |        |
|  |        |--- device n --> sample test matrix --> alls-green verifies 12-20 tests in matrix
|  |
|   simulation-testing matrix
|            |
|            |--- simulator32 --> sample test matrix --> alls-green verifies 12-20 tests in matrix
|            |
|            |--- simulator64 --> sample test matrix --> alls-green verifies 12-20 tests in matrix
|
linux-testing
     |
     |--> sample test matrix --> alls-green verifies 12-20 tests in matrix

Effectively we're running nested matrix tests. We want to run a verification step of each nested matrix because some of them are required to pass, and others are not. So you can see that the inner-most matrix of tests is followed by a separate call to alls-green.

edit:

• for reference: part of the implementation is found here: CI: add verification job to existing parallel build tests golioth/golioth-firmware-sdk#884

[webknjaz]

We want to run a verification step of each nested matrix because some of them are required to pass, and others are not. So you can see that the inner-most matrix of tests is followed by a separate call to alls-green.

Interesting.. The way I envisioned this originally was that the matrices would make use of continue-on-error. Are you trying to use alls-green just to extract the statuses into the job summaries? I never considered this case because there's not really enough visibility into the matrix jobs through the ${{ needs }} context (I imagine it might be possible to extract it via API, I guess).
Or do you want to fail reusable workflows via alls-green?

[szczys]

The problem I was running into is that a skipped job is considered success() by GitHub Actions, but we want to ensure that none of the matrix tests are skipped. We're using alls-green to filter out tests that actually ran and were successful from those that were skipped (or failed).