Retry Mechanism for Package Testing

[j1shnu]

Describe the feature:

In environments like mine, where automated infrastructure changes (like package installation/removal) occur frequently or concurrently, Goss tests may incorrectly report failures if they happen to run during transient states (eg: while a package is being uninstalled or reinstalled).

To make Goss more resilient in such situations, I propose adding retry logic for packages testing. This would allow Goss to re-check a resource after a short delay, mitigating false negatives caused by temporary inconsistencies during automation overlaps.

Describe the solution you'd like:

The proposed solution introduces two new optional fields for each resource block:

package:
  httpd:
    retry: true             # Enables retry mechanism
    retry_delay: 180        # Delay (in seconds) before retrying the check
    installed: true
    name: httpd
    versions:
      - 2.2.15
    skip: false

When retry: true is set:

• Goss should perform the initial check.
• If the check fails, Goss should wait for retry_delay seconds and then re-run the check once again before reporting the failure.

This simple addition would be especially useful in setups where:

• Packages are reinstalled periodically.
• Tests may overlap with these operations, leading to race conditions and false alerts.

Additional Notes:

I am willing to work on implementing this feature and contributing it to the project. Before I begin, I’d love to get feedback from the maintainers:

• Any preferred approach for implementing retries for package testing?
• I also would like to implement similar mechanism for DNS testing too.

Looking forward to hearing your thoughts and thanks for maintaining such a useful tool!

[ns-mjames]

@aelsabbahy I think this is a much needed feature, not just in packages but in all the test. We do have a global retry option, but If we need to selectively retry a specific test, this would be much useful. Can we get it done ?

[ns-mjames]

@aelsabbahy could you help with this ?

[j1shnu]

A slight change in the approach I mentioned. I would love to use this mechanism in this way:

package:
  httpd:
    retry_count: 1          # Enables retry mechanism when greater than 0 and retry number of time mentioned here
    retry_delay: 180        # Delay (in seconds) before retrying the check
    installed: true
    name: httpd
    versions:
    - 2.2.15
    skip: false

[j1shnu]

@aelsabbahy I have implemented the same and tested in my environment. Everything is working as expected. Here is the PR:
#1037