Skip to content

feat: Report checks that are not reachable from any profile - #92

Draft
silug wants to merge 1 commit into
simp:masterfrom
silug:feature/orphaned-checks
Draft

feat: Report checks that are not reachable from any profile#92
silug wants to merge 1 commit into
simp:masterfrom
silug:feature/orphaned-checks

Conversation

@silug

@silug silug commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

A check that no profile can select can never be enforced. These accumulate when a benchmark update drops rules but leaves the checks behind, and nothing in scelint currently reports them.

Adds check_orphaned_checks, run after the merged-data pass.

Reachability

Mirrors the correlation rules in ComplianceEngine::Data#mapping?: a direct reference in a profile's checks, a shared control, a shared CE, or a connection through the controls on a CE.

It is deliberately computed from unconfined data rather than by calling #check_mapping. A check that is only reachable under some confinement is still reachable, and the confined view makes the answer depend on facts, enforcement tolerance, and environment data.

That is not hypothetical. test_module_01's 01_confine_in_ces maps to a CE confined by module_name, which resolves away when no environment data is loaded. The confined view reports it as an orphan; the unconfined view correctly does not.

Severity

Notes, not warnings — a module may legitimately ship checks that only profiles in another module map. Skipped entirely when no profiles are loaded, since #validate already reports that case.

This matters in practice: simp_enterprise_el_disa reports 177 orphans on its own, because much of what it ships is mapped elsewhere. simp_enterprise_windows_cis, which is self-contained, reports 0 after cleanup. Running scelint across the whole modulepath is what gives a trustworthy answer, and that is called out in the method docs.

Happy to reconsider the severity, or gate it behind an option, if 177 notes on a single module is judged too noisy even as info-level output.

Validation

Against simp_enterprise_windows_cis, before and after the cleanup in sicura-nx!3758:

orphans
before 37
after 0

37 matches that MR's own count exactly. The one orphan it deliberately kept — reachable through add_simp_enterprise_windows_cis_to_simp_classes — is correctly not reported, without any special-casing.

Performance

unconfined and controls_for are memoized, and single-fragment components skip the defensive deep copy. The check runs in ~1.8s on simp_enterprise_el_disa (669 checks, 2077 CEs, 9 profiles), the largest module I have.

Draft

Marked draft pending a decision on the note-vs-warning severity above.

Verification

  • rake spec: 104 examples, 0 failures, 1 pending (pre-existing)
  • rubocop: 12 files, no offenses
  • new fixture test_module_14 reports 2 orphans (one via an unmapped CE, one with no references at all) and 1 reachable check

A check that no profile can select can never be enforced.  These
accumulate when a benchmark update drops rules but leaves the checks
behind, and nothing currently reports them.

Reachability mirrors the correlation rules in
ComplianceEngine::Data#mapping?: a direct reference in a profile's
'checks', a shared control, a shared CE, or a connection through the
controls on a CE.

Reachability is computed from unconfined data rather than through
#check_mapping.  A check reachable only under some confinement is still
reachable, and using the confined view makes the result depend on facts,
enforcement tolerance and environment data.  Concretely, test_module_01's
01_confine_in_ces maps to a CE confined by module_name, which resolves
away when no environment data is loaded; the confined view reports it as
an orphan, the unconfined view correctly does not.

Reported as notes, not warnings: a module may legitimately ship checks
that only profiles in another module map.  Skipped entirely when no
profiles are loaded, since #validate already reports that.

Validated against simp_enterprise_windows_cis: 37 orphans before the
cleanup in sicura-nx!3758, 0 after, matching that MR's own count.  The
one orphan it deliberately kept is reachable through a control and is
correctly not reported.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@op-ct op-ct added this to Org Triage Aug 25, 2026
@silug silug moved this from New to Todo in Org Triage Aug 27, 2026
@simp-automation simp-automation Bot moved this from Todo to In Progress in Org Triage Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

2 participants