feat(hdf-converters): add Semgrep JSON to HDF mapper - #8571
Open
clem-field wants to merge 1 commit into
Open
Conversation
Semgrep's native JSON carries a rule metadata layer that its SARIF output
drops: impact and likelihood, the ASVS control mapping, reference URLs,
vulnerability_class, technology and subcategory, and the bandit rule
cross-reference. SARIF keeps CWE, OWASP and confidence only as untyped prose
tags on the rule object. Converting the native format preserves the
cross-framework links that make the results usable for accreditation.
One control per rule: Semgrep metadata is rule-scoped and identical across
occurrences, so findings collapse into results under a single control keyed
on check_id.
Notes on the format, all confirmed against semgrep 1.174.0 output rather
than its documentation:
- metadata fields documented as arrays arrive as bare strings when a rule
declares a single value (owasp, subcategory, technology); everything
list-shaped is normalized on read
- CWEs are emitted in prose form ('CWE-89: Improper Neutralization of ...'),
so the id is parsed out before the NIST lookup
- extra.lines and extra.fingerprint are redacted to the literal string
'requires login' in unauthenticated scans and are filtered rather than
mapped
- metadata.impact rates the severity of the consequence and is not HDF's
impact float; it is tagged as semgrep_impact so it cannot shadow it
- findings suppressed with a nosemgrep comment are omitted from the output
entirely rather than flagged, so no skipped status is derivable
- extra.fix is replacement text for the matched span, not a standalone
instruction, and is labelled accordingly
Scan errors become their own control with status error, present only when
the scan produced any, so a file that failed to parse is visible rather
than buried in passthrough.
The fingerprint keys on semgrep-specific paths (paths.scanned,
engine_requested, skipped_rules) rather than the generic results/errors/
version triple, which matched bare arrays of HDF controls belonging to
other converters.
Also normalizes quoting on the adjacent CHECKOV fingerprint line to satisfy
the repository prettier configuration.
Signed-off-by: clem-field <kc8yhe@me.com>
clem-field
force-pushed
the
feature/semgrep2hdf
branch
from
August 23, 2026 12:41
db13078 to
98bff5f
Compare
|
Contributor
|
This pull request has a conflict. Could you fix it @clem-field? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Adds a mapper for the native
semgrep scan --jsonformat, plus its fingerprint, frontend intake wiring, sample data and spec.Why not just use sarif2hdf
Semgrep emits SARIF, so
sarif2hdfcovers it in principle. But SARIF carries Semgrep's rule metadata only as untyped prose tags on the rule object ("CWE-939: ...","LOW CONFIDENCE") and drops the rest outright:impact,likelihood, the ASVS control mapping,references[],vulnerability_class,technologyandsubcategory.Those are the cross-framework links that make results usable for accreditation — the ASVS
control_idin particular — so the native format gets its own mapper.Granularity
One control per rule. Semgrep metadata is rule-scoped and identical across occurrences, so only the location varies and occurrences collapse into results under a single control keyed on
check_id.Format notes
All of these were confirmed against
semgrep 1.174.0output rather than its documentation, which is incomplete on each point:owasp,subcategory,technology,references); everything list-shaped is normalized on read"CWE-89: Improper Neutralization of ..."), so the id is parsed out before the NIST lookupextra.linesandextra.fingerprintare redacted to the literal string"requires login"in unauthenticated scans, and are filtered rather than mappedmetadata.impactrates the severity of the consequence and is not HDF's impact float; it is taggedsemgrep_impactso it cannot shadow itnosemgrepcomment are omitted from the output entirely rather than flagged, so no skipped status is derivable and every finding is reported as failedextra.fixis replacement text for the matched span, not a standalone instruction — rendering it bare produces messages likeSuggested fix: FalsenameandshortDescription, so a title is derived from the final segment of the dotted rule iderrors[].typeis a heterogeneous array (discriminant plus optional payload) and is read only for its discriminantScan errors
Scan failures become their own control with status
error, present only when the scan reported any. A file that failed to parse was not fully analyzed, so absence of findings in it is not evidence of compliance — burying that in passthrough makes it invisible in Heimdall.Fingerprint
Keys on semgrep-specific paths (
paths.scanned,engine_requested,skipped_rules) rather than the genericresults/errors/versiontriple. The generic form matched bare arrays of HDF controls and would have hijacked input belonging to other converters; verified against all 135 sample files insample_jsons/, with no collisions in either direction.Testing
13 tests covering rule grouping, severity mapping, CWE-derived NIST tags, the string-or-array
owaspnormalization, thesemgrep_impactrename, refs shape, the redacted-placeholder filter, the empty scan, and the scan-errors control.vitest runinlibs/hdf-converters: 167 passing. The 4 failures (3 sonarqube, 1 splunk reverse) reproduce identically on an unmodifiedmastercheckout.