feat: Report checks that are not reachable from any profile - #92
Draft
silug wants to merge 1 commit into
Draft
Conversation
A check that no profile can select can never be enforced. These accumulate when a benchmark update drops rules but leaves the checks behind, and nothing currently reports them. Reachability mirrors the correlation rules in ComplianceEngine::Data#mapping?: a direct reference in a profile's 'checks', a shared control, a shared CE, or a connection through the controls on a CE. Reachability is computed from unconfined data rather than through #check_mapping. A check reachable only under some confinement is still reachable, and using the confined view makes the result depend on facts, enforcement tolerance and environment data. Concretely, test_module_01's 01_confine_in_ces maps to a CE confined by module_name, which resolves away when no environment data is loaded; the confined view reports it as an orphan, the unconfined view correctly does not. Reported as notes, not warnings: a module may legitimately ship checks that only profiles in another module map. Skipped entirely when no profiles are loaded, since #validate already reports that. Validated against simp_enterprise_windows_cis: 37 orphans before the cleanup in sicura-nx!3758, 0 after, matching that MR's own count. The one orphan it deliberately kept is reachable through a control and is correctly not reported. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A check that no profile can select can never be enforced. These accumulate when a benchmark update drops rules but leaves the checks behind, and nothing in scelint currently reports them.
Adds
check_orphaned_checks, run after the merged-data pass.Reachability
Mirrors the correlation rules in
ComplianceEngine::Data#mapping?: a direct reference in a profile'schecks, a shared control, a shared CE, or a connection through the controls on a CE.It is deliberately computed from unconfined data rather than by calling
#check_mapping. A check that is only reachable under some confinement is still reachable, and the confined view makes the answer depend on facts, enforcement tolerance, and environment data.That is not hypothetical.
test_module_01's01_confine_in_cesmaps to a CE confined bymodule_name, which resolves away when no environment data is loaded. The confined view reports it as an orphan; the unconfined view correctly does not.Severity
Notes, not warnings — a module may legitimately ship checks that only profiles in another module map. Skipped entirely when no profiles are loaded, since
#validatealready reports that case.This matters in practice:
simp_enterprise_el_disareports 177 orphans on its own, because much of what it ships is mapped elsewhere.simp_enterprise_windows_cis, which is self-contained, reports 0 after cleanup. Running scelint across the whole modulepath is what gives a trustworthy answer, and that is called out in the method docs.Happy to reconsider the severity, or gate it behind an option, if 177 notes on a single module is judged too noisy even as info-level output.
Validation
Against
simp_enterprise_windows_cis, before and after the cleanup insicura-nx!3758:37 matches that MR's own count exactly. The one orphan it deliberately kept — reachable through
add_simp_enterprise_windows_cis_to_simp_classes— is correctly not reported, without any special-casing.Performance
unconfinedandcontrols_forare memoized, and single-fragment components skip the defensive deep copy. The check runs in ~1.8s onsimp_enterprise_el_disa(669 checks, 2077 CEs, 9 profiles), the largest module I have.Draft
Marked draft pending a decision on the note-vs-warning severity above.
Verification
rake spec: 104 examples, 0 failures, 1 pending (pre-existing)rubocop: 12 files, no offensestest_module_14reports 2 orphans (one via an unmapped CE, one with no references at all) and 1 reachable check