fix(atlas): reconcile MITRE ATLAS framework data v4.0 -> v5.6.0 (#187) - #211
Merged
Conversation
…187) The ATLAS layer in data/frameworks/mitre-atlas.json was seeded from ATLAS v4.0 (per its changelog). ATLAS has since done its major "ML"->"AI" rename plus a significant technique renumber, so ~half of the 50 entries were broken: 6 cited IDs were removed upstream (the #187 trigger) and ~17 more still "existed" but now point to entirely different techniques (e.g. repo T0048="Discover ML Model Family" but upstream T0048=External Harms; T0057="LLM Plugin Compromise" but upstream T0057=LLM Data Leakage). Reconciled every entry by intent against current ATLAS (v5.6.0, mitre-atlas/atlas-data dist/ATLAS.yaml). All 43 resulting IDs are verified present upstream and titles match the authoritative names (the two Prompt-Injection sub-techniques keep the clearer "Direct/Indirect Prompt Injection" labels). Notable decisions (maintainer-approved): - 6 removed IDs: T0022->T0012 (Valid Accounts), T0027->T0024.001 (Invert AI Model), T0030->T0024 + T0057 (Exfiltration via AI Inference API / LLM Data Leakage), T0032->T0070 (RAG Poisoning), T0045->T0048.002 (Societal Harm); T0039 lives only in the generated incidents feed (fixed upstream). - Renumbered/merged: T0035->T0029, T0048->T0014, T0053->T0013, T0056->T0057, T0021->T0057, T0041->T0056, T0037->T0067, T0002->T0006, T0003->T0002, T0004->T0000, T0055->T0048, T0057->T0011.002, T0063->T0071, T0068->T0035. - Dropped 2 tactic-level umbrellas (T0000 "ML Model Access", T0001 "ML Attack Staging") that aren't techniques and are covered by T0040/T0044. Custom GenAI descriptions preserved (concatenated on merge). Regenerated docs/frameworks-registry.js. Follow-ups (separate, content-review PRs): the hybrid markdown ATLAS cross-references, the classifier split labels, and the upstream genai_incidents T0039.
Contributor
🛡️ OWASP GenAI Security CrosswalkThis PR modifies 1 AI-related file(s). The following OWASP GenAI risks may be relevant:
🔍 Scanned 2 changed files · Found 15 potential risks · Full crosswalk · Gap analysis |
This was referenced Jun 9, 2026
emmanuelgjr
added a commit
that referenced
this pull request
Jun 9, 2026
#187) (#212) Follow-up to the data-file reconciliation (PR #211). The markdown ATLAS layer was a *hybrid* — most entries already used current ATLAS, but a handful still carried v4.0 IDs that now point at different techniques. Reconciled by NAME (the *_MITREATLAS.md detail tables carry technique names, making intent unambiguous); custom GenAI labels that sit on stable IDs (T0040/T0015/T0025/T0035) were left as the author's deliberate mappings. ID corrections (verified against ATLAS v5.6.0 / atlas-data dist/ATLAS.yaml): - T0037 -> T0067 (Output Manipulation; T0037 is now "Data from Local System") - T0063 -> T0071 (Embedding Manipulation; T0063 is now "Discover AI Model Outputs") - T0041 -> T0056 (System-Prompt Leakage refs; T0041 is now "Physical Environment Access") - T0032 -> T0070 (RAG/data poisoning; T0032 removed upstream) - T0045 -> T0048.002 (Disinformation -> Societal Harm; T0045 removed upstream) - T0010.000 -> T0010 and T0016.000 -> T0016 (sub-technique refs used where the parent "AI Supply Chain Compromise" / "Obtain Capabilities" was meant) - T0021 -> T0057 under LLM02 only (Sensitive Information Disclosure = LLM Data Leakage; Agentic's T0021 = "Establish Accounts" is correct and left as-is) - DSGAI_MITREATLAS detail row "Obtain Capabilities" mislabeled AML.T0012 (Valid Accounts) -> AML.T0016 29 mapping files + regenerated entries/backlinks/data bundle. validate.js 0/0/385; markdownlint clean. frameworks-registry.js intentionally untouched (owned by #211). Remaining: classifier split labels + upstream T0039.
emmanuelgjr
added a commit
that referenced
this pull request
Jun 10, 2026
…nts (#187) (#214) Final two tails of #187: 1. classifier/splits relabel (calibration.json, test.json, split_meta.json) The splits were frozen at commit 10f2372, before the ATLAS v4.0->current reconciliation (PRs #211/#212), leaving 25 ATLAS rows (2 calibration, 23 test) pointing at IDs that are stale in current ground truth. Each stale control_id was remapped by positional alignment of its entry's MITRE ATLAS mapping list between the freeze commit and HEAD - i.e. the maintainer's own curation history decides the successor, not fresh judgment (verified consistent with the #211/#212 decision tables, e.g. T0037->T0067, T0063->T0071, T0045->T0060, T0022->T0021, T0068->T0053). control_name refreshed from current entries on all 117 ATLAS rows; hashes recomputed per the documented {entry_id}::{framework}:: {control_id} format. Split membership, counts, severity/tier/scope/ notes (eval-query inputs), and all non-ATLAS rows are unchanged. Provenance recorded in split_meta.json under "relabels". Invariants verified: calibration/test hash-disjoint (no leakage), no duplicate ATLAS rows, every ATLAS row present in current data/entries ground truth AND in current upstream dist/ATLAS.yaml (117/117). 2. Incidents sync (data/incidents.json, docs/incidents.js) Pulled upstream genai_incidents (emmanuelgjr/genai_incidents#26), which removed the phantom AML.T0039 - an ID that never existed in any ATLAS release - from the corpus. 12 of our 125 curated incidents carried it; all already cite AML.T0048 (External Harms), so no signal lost. validate.js: 0 errors / 0 warnings / 385 passed. Closes #187.
emmanuelgjr
added a commit
that referenced
this pull request
Jun 10, 2026
…ion (#215) Counts drifted after the ATLAS v4.0->current reconciliation (#211: 50->43 ATLAS entries) and the incidents-feed work: - controls 1,514 -> 1,507 (README badge + 5 text spots, CITATION.cff, banner.svg, og-image.svg) - incidents 114 -> 125 (banner.svg, og-image.svg) - mapping files 67 -> 70 (banner.svg engine box + stat card, og-image.svg) - mappings 3,210 -> 3,351 (banner.svg) All figures verified against the data layer (data/frameworks/*.json, data/entries/*.json, data/incidents.json, source md dirs). Historical records (CHANGELOG, EVAL_REPORT, web-app timeline) left as-is. validate.js PASSED; markdownlint clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Re-triaging #187 (the 6 removed ATLAS IDs) revealed the problem is much larger:
data/frameworks/mitre-atlas.jsonwas seeded from ATLAS v4.0 (per its changelog), and ATLAS has since done its major "ML"→"AI" rename + technique renumber. Of the 50 entries:T0048="Discover ML Model Family" but upstreamT0048=External Harms;T0057="LLM Plugin Compromise" but upstream=LLM Data Leakage;T0021="Data Leakage" but upstream=Establish Accounts). The validator/diff never flagged these because the ID strings are valid.Fixing only the 6 would have left ~17 entries pointing at the wrong technique while looking fixed.
What this PR does
Reconciles every entry by intent against current ATLAS (v5.6.0,
mitre-atlas/atlas-data→dist/ATLAS.yaml). Result: 50 → 43 entries, all verified present upstream with authoritative titles (the two prompt-injection sub-techniques keep the clearerDirect/Indirect Prompt Injectionlabels). Custom GenAI descriptions are preserved (concatenated where entries merge). Regenerateddocs/frameworks-registry.js.Key remaps (maintainer-approved)
Also bumped
version4.0 → 5.6.0 andlast_synced.Verification
commagainstdist/ATLAS.yaml= empty residual).node scripts/validate.js→ 0 errors / 0 warnings / 385 passed.Deliberately scoped out (separate content-review PRs to follow)
classifier/splits/*.jsonlabels — ML training/test data (5 dead-ID labels + hybrid).data/incidents.json/T0039— generated from upstreamgenai_incidents; fix belongs upstream.Closes the data-file portion of #187.