Rule Testing Infrastructure

[evankanderson]

Please describe the enhancement

Significantly enhance the rule testing and Continuous Integration (CI) tooling for Minder's security policies.

Taken from the LFX Mentorship proposal

The successful completion of this project will provide Minder users with a robust, automated, and maintainable framework for ensuring the quality and correctness of custom security rules.

• Improve Rule Testing Tools: Develop a standalone testing command for Minder rules.
• Robust Testing: Allow for the definition and execution of multiple distinct test cases against a single rule.
• Focus on Local Execution: Tests should be able to execute locally without requiring network access to external resources.
• Support Diverse Data Ingestion: Design the framework to accommodate tests for rules covering all current data ingestion methods: REST API, Git repositories, and declared data sources.
• Provide Automated CI Tooling: Implement CI workflows to automatically detect and run tests.
• Update Existing Rules: Migrate and update existing tests within the github.com/mindersec/minder-rules-and-profiles repository to utilize the new framework.
• Documentation and UX: Create documentation for the new tooling and address potential client-side UX issues related to testing.

Solution Proposal

TODO (part of the LFX mentorship to design the solution)

Describe alternatives you've considered

No response

Additional context

No response

Acceptance Criteria

Following documentation on the https://docs.mindersec.dev/ website, it should be clear how to set up automated testing for any Minder rules using a GitHub Action.

• The Action should not need to check out or build the Minder source code, and should use (and verify) a released artifact
• The test command should exit 0 on all tests passing (e.g. standard Unix semantics)
• Command output from failed tests should be clear and enable easy debugging and fixing of the rule
• It should be easy to create and store realistic or simplified rule inputs e.g. from gh api output
• Testing should support all our existing input types, including data sources as well as REST and Git ingestion
  • It should be easy and clear how to extend the test command for new ingestion types as they are written

[evankanderson]

Will assign this to @krrish175-byte when he's added to the mindersec org. Expected start is June 1.

[evankanderson]

Is the sanitization layer primarily deterministic/rule-based or model-assisted?

I'm not sure what sanitization layer you're referring to (I don't see it in the issue). Can you explain further what you're asking about?

[evankanderson]

This addition to the Minder project is currently on the roadmap, scheduled to be designed and implemented this summer as part of an LFX mentorship (linked in the header of the issue).

I'm a bit confused about your angle -- this is issue 6434 in an existing open source repository -- the tech stack, backend language are implicit in the repo context, and the problems solved are described in the issue.

Given that this is an open source project, I don't think choosing private non-project communication channels (e.g. not GitHub issues or OpenSSF Slack) is appropriate.

[krrish175-byte]

Design Document - #6492