ScoutingVVVTools

CMS scouting VVV analysis workflow for pileup reweighting, branch conversion with jet energy corrections, optional sample-entry mixing, BDT/NN training, signal-region optimization, QCD ABCD estimation, systematic uncertainties, data/MC plotting, and CMS combine significance/limits.

Pipeline order

1. mode=1: build pileup-weight CSV files.
2. mode=0 with selections/convert_ttbar_cr/config.json, then selections/jms_jmr/jms_jmr_fit.py: convert the semileptonic ttbar control region with the same jet corrections (without JMS/JMR) and fit the AK8 soft-drop mass scale and resolution (JMS/JMR) that the signal-region conversion applies.
3. mode=0: convert the selected NanoAOD-style inputs into the five analysis trees boosted3, boosted2_resolved, boosted2_unresolved, boosted2_leptonic_1lep, and boosted2_leptonic_2lep, with the official jet energy corrections, Type-1 MET, JMS/JMR, and, for MC, one <tree>__<variation> tree per jet systematic variation in the same files (see Jet energy corrections).
4. mode=6: shuffle each selected MC sample across its ROOT chunks while preserving chunk sizes and filenames, using a deterministic block shuffle that keeps ROOT reads mostly sequential.
5. mode=2: train the BDT or PyTorch NN and save the model plus copied configs. The checked-in config reads the unmixed converted dataset (input_pattern = {input_root}/{sample_group}/{sample}.root) and uses model_type: "bdt".
6. mode=3: scan the test split and write signal_region.csv.
7. mode=5: run the QCD ABCD validation on the MC test split and write the combine inputs.
8. mode=8, mode=9, mode=11, mode=12, mode=13, mode=14: theory, pileup, JES, JER, JMS, and JMR yield ratios per MC sample, tree, and signal region.
9. mode=4: make data/MC comparison plots.
10. mode=7: run the CMS combine wrapper to compute expected significance and AsymptoticLimits per signal class / signal sample / combined, both with the MC-true QCD class yields and with the merged ABCD QCD prediction, with every systematic uncertainty of Systematic uncertainties.

selections/b_veto/ and systematics/ are separate studies and are not launched through run.py.

run.py

Usage:

python3 run.py <mode> [config.json] [sample1 sample2 ...] [--slurm ...]

run.py is the job dispatcher; the older run.sh does not link correctionlib and cannot build the current convert_branch.C. For the C++ modes it compiles the tool in its directory (auto-detecting OpenMP, and correctionlib through python3 -c "import correctionlib", whose library directory is embedded as an rpath) and, for local runs, deletes the compiled binary at exit; selections/convert/convert_branch is a tracked file, so build test binaries elsewhere. Logs go to log.txt in each mode's directory (or log_<config stem>.txt for a non-default config).

Examples:

python3 run.py 1
python3 run.py 0
python3 run.py 0 selections/convert/config.json www qcd_ht2000
python3 run.py 0 selections/convert_ttbar_cr/config.json
python3 run.py 6
python3 run.py 2
python3 run.py 3 selections/signal_region/config.json
python3 run.py 5 background_estimation/config.json
python3 run.py 8
python3 run.py 11
python3 run.py 4 plotting/config.json
python3 run.py 7 combine/config.json

Modes:

• mode=0: compile and run selections/convert/convert_branch.C. Input: selections/convert/config.json, selections/convert/branch.json, selections/convert/selection.json, src/sample.json, the pileup CSVs from mode=1, the jet-correction files of the jet_pt_correction block, and the JMS/JMR fit result. Output: converted ROOT files under {output_root}/{signal|bkg|data}/, plus selections/convert/log.txt. Each file holds the five nominal analysis trees and, for MC, the <tree>__<variation> jet-systematic trees (see Jet energy corrections). run.py mode=0 asks convert_branch.C for the batch count and invokes it once per batch, so each program invocation reads at most actual_thread_count * 32 input files (or CONVERT_FILES_PER_BATCH; --slurm-files-per-job for SLURM). Each batch is written first to {output_root}/{sample_group}_tmp/{sample}_X.root; the final batch updates raw_entries once using the whole-sample entry total accumulated across batch bookkeeping and then merges those batch files into the normal final output_pattern while still honoring max_output_file_size_gb. resume_successful_batches defaults to true; when enabled, mode 0 requires exactly one actually selected sample and skips only existing batch outputs whose ROOT file opens with every configured output tree and whose .raw_entries sidecar is readable, preserving those files for the final merge. Remote ROOT input open failures are retried five times before the batch fails and the run continues to the next batch. Convert resolves configured output, pileup, branch, and selection paths to absolute paths at startup to reduce sensitivity to stale cwd state in long AFS/NFS jobs. The final writer streams each tree through a TChain clone owned by the chain itself, so branch addresses stay correct when ROOT advances across temp-file boundaries, and it aborts on any chunk/tree entry-count mismatch instead of silently writing too few or repeated entries. The convert expression engine includes collection helpers such as sum, max_value, min_value, nth_max_value(collection, expr, rank, default), value_at_max(collection, key_expr, value_expr, default), and value_at_nth_max(collection, key_expr, value_expr, rank, default); the checked-in output branch config writes per-AK8 WvsQCD and VvsQCD scores, event-level ScoutingFatPFJetRecluster_{WvsQCD,VvsQCD}_{sum,max} summaries, and AK4 b-tag summaries including ScoutingPFJetRecluster2_scoutUParT_probb_max_pt, ScoutingPFJetRecluster2_scoutUParT_probb_sum, ScoutingPFJetRecluster2_scoutUParT_probb_second_max, and ScoutingPFJetRecluster2_scoutUParT_probb_second_max_pt for every tree. selections/convert/check_conversion.py checks the batch and merged outputs, including every configured variation tree of an MC sample and its listed branches. The pileup root files can be obtained from using wget on these links: https://cms-service-dqmdc.web.cern.ch/CAF/certification/Collisions24/PileUp/dataPileupHistogram-2024CDEFGHI_Golden-66000ub.root, https://cms-service-dqmdc.web.cern.ch/CAF/certification/Collisions24/PileUp/dataPileupHistogram-2024CDEFGHI_Golden-69200ub.root, https://cms-service-dqmdc.web.cern.ch/CAF/certification/Collisions24/PileUp/dataPileupHistogram-2024CDEFGHI_Golden-72400ub.root

• mode=1: compile and run selections/weight/weight.C. Input: selections/weight/config.json and src/sample.json. Output: pileup CSV files under the configured output_root, plus selections/weight/log.txt.

• mode=6: compile and run selections/mix/mix.C. Input: selections/mix/config.json, src/sample.json, and the converted MC ROOT files from mode=0. Output: shuffled ROOT files under the configured output_root/{signal_mixed|bkg_mixed}/, preserving the original chunk filenames and per-chunk entry counts while applying a deterministic per-tree block shuffle for each selected MC sample, plus selections/mix/log.txt. Missing input files for a requested sample are fatal. The implementation scans input chunk entry counts first (warming ROOT serially, then using OpenMP across the remaining files when available) and then rewrites each tree through a TChain using shuffled contiguous entry blocks plus an in-block cyclic rotation. Each non-empty output chunk is cloned from the TChain itself so ROOT keeps branch addresses synced across input-file boundaries, and the writer validates both per-chunk and total entry counts before closing. The block size is chosen as clamp(total_entries / 512, min_block_entries, max_block_entries); the checked-in defaults are 32 and 4096. This keeps the output format unchanged while avoiding the old fully random entry-by-entry ROOT read pattern.

• mode=2: run selections/BDT/train.py. Input: selections/BDT/config.json, the ROOT files resolved from its input_root / input_pattern (the checked-in config points to the mixed mode=6 output under ../../dataset/{sample_group}_mixed), and src/sample.json. Output: one trained model directory per tree under output_root. With model_type: "bdt" it contains {tree}_model.json and {tree}_model_stage1.json; with model_type: "nn" it contains PyTorch checkpoints {tree}_model.pt and {tree}_model_stage1.pt and requires PyTorch at training/inference time. The rest of the output layout is shared: feature_corr.pdf, loss plots (loss_mlogloss.pdf / loss_classification.pdf / loss_total.pdf for BDT or loss_weighted_ce.pdf / loss_objective.pdf for NN, plus loss_decorrelation.pdf; NN x-axes are epochs, BDT x-axes are boosting rounds), stage-tagged comparison plots such as importance_cls.pdf / importance_decorr.pdf, roc_*_cls.pdf / roc_*_decorr.pdf, score_*_cls.pdf / score_*_decorr.pdf, decor_corr_{train,test}_cls.pdf / decor_corr_{train,test}_decorr.pdf, decor_score_vs_branch_cls.pdf / decor_score_vs_branch_decorr.pdf, and decor_branch_shapes_by_signal_score_cls.pdf / decor_branch_shapes_by_signal_score_decorr.pdf. BDT feature importance uses XGBoost gain; NN feature importance uses deterministic permutation importance while preserving the same filenames. The output also includes a branches/ subdirectory with one normalized per-class input-distribution PDF per training branch, copied config.json / branch.json / selection.json, test_ranges.json, and the saved test-set prediction references test_reference_signal_region.npz, test_reference_qcd_est.npz, and test_reference_qcd_est_full.npz, plus selections/BDT/log.txt. Both model types keep the same sample splitting, clipping/log preprocessing, event-weight construction, dynamic learning-rate reduction, prediction-reference validation, and downstream score interface. Per-tree qcd_ht_training_weight_step scales only the internal train.py training/evaluation weight for qcd_ht* samples in increasing HT order before the existing class balancing; saved prediction-reference weights keep the original physics weight_physics. When a per-tree decorrelate branch also appears in selection.json thresholds, that branch's threshold is normally omitted from the training/early-stopping objective splits and their class balancing; for decorrelate branches named ScoutingFatPFJetRecluster_msoftdrop_*, the training view instead applies a loose (0, decor_msoftdrop_training_max) window when that branch has a selection threshold. Ordinary branch, ROC, score, feature-importance, feature-correlation, and downstream reference outputs still use the full threshold set, while decorrelation diagnostics use the training-threshold view. BDT logs sum-scale classification_loss and stage 1 monitors test classification_loss; NN uses class-balanced mini-batches with sampling-corrected event weights for backward() and logs epoch-end eval-mode train_objective_loss / test_objective_loss, where stage 1 uses objective_loss = weighted_CE and stage 2 uses objective_loss = weighted_CE + scaled_smooth_CvM_decorrelation. NN early stopping monitors epoch-end test objective_loss in both stages, and AdamW weight_decay remains native decoupled optimizer behavior rather than a logged loss term. In NN mode, per-tree {tree}_nn blocks (e.g. boosted3_nn) configure the PyTorch MLP (hidden_layers, dropout, batch_norm, batch_size, learning rates, weight decay, epochs, and permutation-importance event count); batch_size is the total size of each class-balanced mini-batch, the checked-in NN blocks set batch_norm: false, and smooth-CvM decorrelation is supported in the second NN stage with decorrelation scale calibrated against the stage-1 batch-averaged weighted CE.

• mode=3: run selections/signal_region/signal_region.py. Input: selections/signal_region/config.json and one trained model directory from mode=2. Output: sr_score_*.pdf, scores_no_regions.pdf, scores.pdf, scores_no_regions_radial_equalized.pdf, scores_radial_equalized.pdf, and signal_region.csv inside the configured output_dir, plus selections/signal_region/log.txt. The 2D score PDFs use the shared class palette, hide coordinate axes/ticks/frames, and include both the original regular-polygon simplex projection and a radial-equalized projection that keeps polygon vertices fixed while spreading points outward. The script builds one shared candidate pool of general high-dimensional rectangles over the configured BDT score_axes (all in the checked-in config), where every score axis has an independent [low, high) interval and can therefore represent simultaneous lower and upper cuts. Candidate generation combines dense per-axis boundary sets from total/signal/background weighted quantiles plus tail-heavy quantiles, explicit low/mid/high-tail single-axis seeds from seed_quantiles, exact single-axis bounded interval seeds, optional bounded multi-axis seed combinations, deterministic beam coordinate search where every per-axis update scans exact [edges[a], edges[b]) intervals, compatibility expansion that optimizes regions constrained to be non-overlapping with high-Z anchor candidates, and local event-threshold refinement. Independent edge building, beam updates, compatibility updates, local-refinement workers, and event-mask canonicalization can use max_threads, but each stage merges results in deterministic scan order so the candidate set is unchanged by threading. No candidate-pool or global-candidate count limit is applied; candidates are removed only by rounded geometry duplicates and by final exact event-mask/canonical-box deduplication. Before global selection, every retained candidate is shrunk to an event-preserving minimal score box; candidates are deduped only when they select the same exact event mask and shrink to the same canonical box. The final selection is global rather than sequential: it searches K mutually non-overlapping candidates and maximizes the existing combined objective sqrt(sum Z_i^2), with Python and OpenMP compatibility checks using the same exact half-open [low, high) overlap semantics as event-mask membership. The OpenMP helper first finds a beam incumbent and then runs branch-and-bound; if the search completes, the result is exact for the finite candidate list, while node caps produce a conservative upper-bound certificate. After the K non-overlapping SRs are selected, an empty-bin expansion step runs in order SR1, SR2, …: each SR has its signal-class score-axis upper bounds pushed toward 1.0 and its background-class score-axis lower bounds pushed toward 0.0, but only into space that is empty in MC under the SR's other-axis cuts and only as far as the expanded box remains geometrically non-overlapping with every other selected SR. The expansion is constrained to keep each SR's exact selected event mask, so per-bin S/B/Z are numerically unchanged; only the empty-side score-axis bounds in signal_region.csv are widened. Before scanning, the script reloads the saved model, reproduces the saved full-threshold signal-region test prediction, and aborts if it does not match test_reference_signal_region.npz within the stored tolerances.

• mode=4: run plotting/data_mc.py. Input: plotting/config.json, plotting/branch.json, the converted ROOT files from mode=0, and one trained model directory from mode=2. Output: one physical-normalization PDF per plotted branch under the configured output_root, plus a matching {tree_name}_{branch}_normal.pdf where the data histogram integral and total stacked MC histogram integral are each scaled to 1 with A.U. on the y-axis, plus plotting/log.txt. Ordinary ROOT branches are discovered from the convert output config and are not limited to the BDT input branch list. MC uses the configured trained-model input pattern for ordinary ROOT branches and reads at most the copied BDT config's entries_per_sample entries per sample while keeping the same full-sample normalization. Data reads all configured entries from the non-mixed data directory: if the copied pattern contains {sample_group}_mixed, data resolves it as {sample_group} and therefore uses data, not data_mixed. Missing sample files, empty MC trees/classes before filtering, and non-positive MC raw_entries are treated as fatal errors. In addition to ROOT branches, one derived model score branch per class is plotted as score_{class_name}. For those score branches, MC is read from bdt_root/test_ranges.json, validated against test_reference_signal_region.npz, and normalised to the same full-sample target total while data is predicted from the full configured data_samples input. The script loads .json/.pkl BDT models or .pt NN checkpoints according to the copied model_type.

• mode=5: run background_estimation/qcd_est.py. Input: background_estimation/config.json, one trained model directory from mode=2, and the signal_region.csv written by mode=3. Output: ABCD summary PDFs and one ROOT file under the configured output_dir, plus background_estimation/log.txt. The validation PDFs keep the finite-MC / ABCD-propagated uncertainties based on sum(w^2). The qcd_abcd_region_counts.pdf and qcd_abcd_region_fractions.pdf summaries are stacked by the BDT class_groups classes with a shared fixed class palette; additional colors are generated deterministically when there are more classes than base colors, and qcd_abcd_region_counts_linear.pdf saves the same counts with a linear y axis. The QCD-merged summaries (qcd_abcd_region_counts_qcd_merged.pdf, qcd_abcd_region_counts_qcd_merged_linear.pdf, and qcd_abcd_region_fractions_qcd_merged.pdf) combine every class_groups class whose name contains qcd case-insensitively into one QCD stack color and legend entry, leaving non-QCD classes unchanged. Optional per-tree a_region_shape_branches entries add normalized branch-shape diagnostics under output_dir/a_region_shapes/: for each configured branch, qcd_est writes one PDF for every signal bin plus one PDF for the full A-union, with one unit-area weighted step histogram per class and no ABCD prediction scaling; values <= -10 are dropped before binning/normalization so missing-slot values such as -99 do not set the visible range. The ROOT file stores combine-facing bundles for every saved category (samples/*, groups/*, qcd_predict, qcd_true, total_predict, and total_true) as srN/yield, srN/stat_error, and srN/scale_error one-bin histograms plus a category-level covariance_total TH2. The combine-facing convention is stat_error = sqrt(yield), scale_error = 0, and diagonal covariance_total = diag(yield), so the weighted yield is treated as the Poisson event count. Every SR of every category also stores srN/mc_stat_error = sqrt(sum w^2) of the test-split MC, the input of combine's autoMCStats; for qcd_predict (and total_predict) it is the per-SR part of the ABCD MC statistics, from the A-region fractions, and srN/mc_scale_error adds the SR-correlated part from the B/C/D statistics. metadata/abcd_closure (one row) stores raw_scale, raw_scale_error, multiplier, final_scale (the ABCD scale k = pred_union / A_union), final_scale_error, pred_union, pred_union_error, and the QCD qcd_a_union, qcd_b, qcd_c, qcd_d sums with their *_error. QCD classes are identified by class_groups names containing qcd case-insensitively; all samples from those classes are summed for the ABCD A/B/C/D totals, qcd_true, qcd_predict, and total_predict, while the per-class groups/* outputs remain MC-true. The signal-region score axes are detected from the CSV {axis}_low / {axis}_high columns, so the ABCD step can consume either the legacy independent axes or the new all-score-axis regions. The non-score ABCD dimension is configured by required per-tree abcd_branches; every listed branch must have a threshold in the copied BDT selection.json, A/B require all listed branches to pass, C/D require all listed branches to fail, and partial pass/fail events are excluded. Before building the ABCD regions, the script reloads the saved model, validates against test_reference_qcd_est_full.npz when present (falling back to the legacy filtered test_reference_qcd_est.npz), and aborts on mismatch.

• mode=7: compile and run combine/combine.C. Input: combine/config.json (lists one or more channels, each pointing to a qcd_abcd_yields.root from mode=5) plus one trained bdt_root directory from mode=2, and the *_syst_yields.json of modes 8, 9, and 11–14. combine.C reads class_groups from bdt_root/config.json, then resolves that copied config's sample_config the same way as qcd_est.py so the signal/background-class split still comes from sample.json. Group names from class_groups are matched against groups/* in the ROOT file by the same slugified lowercase convention used by qcd_est.py, and QCD classes are the class names containing qcd case-insensitively. Must be executed inside a CMSSW area that has HiggsAnalysis/CombinedLimit built, so that combine and combineCards.py are on $PATH (tested with combine v10.0.2 in CMSSW_14_1_0; a CMSSW_* area inside the repository is ignored by .gitignore). Output: significance.csv, limits.csv, significance_abcd_mc.csv, limits_abcd_mc.csv, significance_by_channel.csv, limits_by_channel.csv, significance_by_channel_abcd_mc.csv, and limits_by_channel_abcd_mc.csv under the configured output_dir, plus combine/log.txt. Each CSV has one row per signal scenario (combined / per-signal-class / per-signal-sample); the per-channel CSVs add a channel column and rerun the same scenarios using one channel at a time. Every channel card is a shape card with one bin per SR (<channel>_sr<N>): one-bin histograms per process carry the rate and, as bin error, the MC statistics sqrt(sum w^2) (srN/mc_stat_error) used by * autoMCStats <mc_stat_threshold> 0 1; rates and the Asimov observation (the sum of the process rates) are also written explicitly, so a process without yield in an SR drops out of that bin. combine runs on that Asimov data (-t -1). The _abcd_mc.csv CSVs replace all QCD classes with the single merged ABCD prediction, process bkg_qcd_abcd (an MC-true QCD class keeps its bkg_<class> name), while keeping the non-QCD processes on their MC-true group/sample blocks. The nuisances are listed in Systematic uncertainties; each enabled nuisance needs its input, and a missing input, a non-positive lnN kappa, or an unknown enabled_nuisances name stops the wrapper with an error. The wrapper validates every required signal sample/group and aborts on missing inputs or invalid shapes instead of regularizing the yields. If use_root_covariance=true (only with the mcstat nuisance disabled, since both model the statistical uncertainty of the predictions), the full per-process covariance between signal regions is additionally injected via an eigen-decomposition of each process's covariance_total, one Gaussian shape nuisance per retained eigenmode. In that optional mode, when rescale_shape_modes_to_positive is true (default), any background process whose nominal yields and covariance are identically zero across all SRs is dropped before writing the datacard, and any eigenmode whose raw ±1σ templates would make a varying bin negative or a total template norm non-positive is shrunk to the largest safe step below that boundary; the datacard shape coefficient is rescaled by 1/a, zero-valued bins are allowed as long as the varied templates stay non-negative and keep strictly positive total norms, and every drop/rescale is written as a warning to combine/log.txt. If a per-sample signal process is identically zero in some channels, those channels are skipped with a warning; if it is identically zero in every channel, the wrapper records 0 significance and inf expected limits for that row and continues. If AsymptoticLimits finishes successfully but its ROOT output still lacks the expected quantiles, the wrapper logs a warning, records 0 for that row's significance and inf for its expected limits in the CSVs, and continues. A work directory is kept under output_dir/work/ with the generated datacards, shape ROOT files, and combine outputs.

• mode=8: run selections/theory_weights/theory_syst.py (PDF, scale, ISR, and FSR ratios of the samples with has_theory_weights, from the nominal trees' theory-weight branches).

• mode=9: run selections/pileup_syst/pileup_syst.py (weight_pu_up / weight_pu_down over weight_pu).

• mode=10: run plotting/class_shapes.py.

• mode=11, mode=12, mode=13, mode=14: run selections/{jes,jer,jms,jmr}_syst/<syst>_syst.py, thin wrappers around selections/jet_syst_common/jet_variation_common.py that compare each <tree>__<syst>_{up,down} tree with its nominal tree in the same converted files. Every syst script writes <name>_syst_yields.{json,csv} under its output_dir with, per MC sample and tree, the inclusive up/down ratios and a regions entry per signal region; its regions are the ABCD A regions of qcd_est.py (the BDT selection.json thresholds, the ABCD window included, with sentinel values failing, and the signal_region.csv score boxes). A region with fewer than min_region_events (10) nominal MC events takes the inclusive ratios and is flagged fallback_to_inclusive: with fewer events the ratio measures the MC fluctuation rather than the variation. Missing files, trees, branches, or configured inputs are errors. merge_shards.py in each jet-syst directory merges per-sample SLURM shard outputs.

Sample arguments:

• Extra sample names are supported only for mode=0, mode=1, and mode=6.
• If no sample names are given, run.py uses submit_samples from the chosen config.
• If submit_samples is empty or missing, all MC samples in src/sample.json are used.

Log archiving:

• After the final [finished …] line is written, run.py copies the per-mode log.txt into the program's configured output directory for local runs.
• Applies to mode=1 (output_root), mode=2 / mode=4 (per-tree output_root expanded over submit_trees), and mode=3 / mode=5 / mode=7 (output_dir). mode=0 (convert_branch) and mode=6 (mix) are intentionally skipped.
• The copy runs on both success and failure; if the resolved output dir does not yet exist (e.g., the run aborted before creating it), the copy is skipped with a warning instead of erroring.

Main JSON files

• src/sample.json: master sample registry with name, path, sample_ID, is_MC, is_signal, optional has_theory_weights, xsection, lumi, and raw_entries. The MC entries (names, cross sections, raw_entries, genweight_mean, weight_pu*_mean) are those of the 2026-10 reconversion; the VH samples keep their dataset-style names such as ZH-Zto2Q-Hto2Wto4Q and WplusH-Wto2Q-Hto2Wto4Q and the cross sections confirmed for this analysis, not the short names and cross sections of ScoutingVVVTools_sf/src/sample.json. data_2024 is the 2024G/H/I data of the analysis; 2024G_UParT_v6 is the 2024G data used for the data/MC plots and the ttbar control region.
• selections/weight/config.json: pileup histogram inputs and pileup-weight output paths.
• selections/convert/config.json: convert-step paths, threading, file-size splitting, resume_successful_batches for single-sample batch resumption, pileup CSV pattern, and the jet_pt_correction block (see Jet energy corrections).
• selections/convert_ttbar_cr/config.json: the ttbar control-region conversion for the JMS/JMR fit; its jet_pt_correction block applies the same corrections with apply_jms_jmr: false and no variations. It converts the 2024G_UParT_v6 data with the golden JSON selections/convert/Cert_Collisions2024_378981_386951_Golden.json (lumi_mask); the data_samples of plotting/config_ttbar_cr.json, which jms_jmr_fit.py reads through samples_from (the fit takes the luminosity from src/sample.json), name the same data.
• selections/mix/config.json: mix-step tree selection, input/output ROOT roots and patterns, sample config, threading for the input-chunk scan, deterministic random_state, and the block-size bounds min_block_entries / max_block_entries (default 32 / 4096).
• selections/convert/selection.json: event preselection (input scalars only, evaluated before the jet corrections), runtime collections, and the tree split into the five analysis trees.
• selections/convert/branch.json: input branches to read and output branches to write.
• selections/BDT/config.json: model inputs, input_root / input_pattern, class groups, training settings, output directories, top-level model_type (bdt or nn, default bdt), top-level decor_loss_mode, decor_lambda, decor_n_bins, decor_n_thresholds, decor_score_tau, decor_bin_tau_scale, decor_msoftdrop_training_max (positive upper edge for the loose (0, max) training window used only when a decorrelate branch named ScoutingFatPFJetRecluster_msoftdrop_* also has a selection.json threshold), per-tree BDT hyperparameters (n_estimators, n_estimators_decorr, max_depth, learning_rate, learning_rate_decorr, optional min_learning_rate / lr_reduce_patience, gamma, reg_lambda, reg_alpha, min_child_weight, subsample, colsample_bytree, optional colsample_bynode, early_stopping_rounds), per-tree NN hyperparameters in {tree}_nn blocks (e.g. boosted3_nn) (epochs, epochs_decorr, hidden_layers, activation, dropout, batch_norm, batch_size, learning_rate, learning_rate_decorr, optional min_learning_rate / lr_reduce_patience, weight_decay, optional grad_clip_norm defaulting off, early_stopping_rounds, permutation_importance_events; checked-in NN values use learning_rate: 0.001, learning_rate_decorr: 0.0005, and min_learning_rate: 0.00002), decorrelate, decor_efficiencies, event_reweight_branches, and per-tree qcd_ht_training_weight_step for additive QCD HT training-weight scaling.
• selections/signal_region/config.json: signal-region scan settings: lumi, n_signal_regions / N, bdt_root, output_dir, score_axes, min_bkg_weight, min_signal_weight, optional entry minima, max_edge_candidates_per_axis, beam_width, top_intervals_per_axis, coordinate_rounds, seed_intervals_per_axis, multi_axis_seed_max_axes, multi_axis_seed_max_seeds, compatibility_seed_anchors, compatibility_seed_rounds, local_refine_rounds, local_refine_neighbor_edges, local_refine_top_candidates, local_refine_diverse_masks, local_refine_candidate_overscan, global_beam_width, branch_bound_max_nodes, branch_bound_time_limit_seconds, deduplicate_event_masks, require_exact_n_regions, max_threads, progress_every_seconds, and seed_quantiles.
• background_estimation/config.json: qcd_est.py settings, including bdt_root, signal_region_csv, output_dir, root_file_name, required per-tree abcd_branches entries whose thresholds are read from the copied BDT selection.json, and optional per-tree a_region_shape_branches entries for normalized per-class branch-shape plots in each signal bin and the A-union.
• plotting/config.json: data_mc.py settings, including bdt_root, output_root, data_samples (["2024G_UParT_v6"]), and per-tree event_reweight_branches (applied to MC events only; data weights stay 1.0). Ordinary MC branch reads use entries_per_sample from the copied BDT config under bdt_root; data reads are not capped and use the non-mixed data directory when the copied BDT pattern points MC at {sample_group}_mixed. Each branch writes both the physical-normalization plot and a _normal.pdf unit-area comparison where data and total MC are separately normalized to 1.
• plotting/branch.json: per-tree plot overrides such as skip_branches, bins, x_range, y_range, logx, and logy; derived model score branches use names like score_VVV and accept the same overrides.
• combine/config.json: combine.C settings: channels (list of {name, root_file, bdt_root} — each ROOT file is the qcd_abcd_yields.root output of mode=5 for one channel, and each bdt_root is that channel's trained tree output directory), output_dir, optional combine_cmd / combine_cards_cmd, use_root_covariance (default false; combine uses binned Poisson statistics from the yields and does not turn ROOT covariance into nuisances), eigen_rel_cutoff (used only when use_root_covariance=true, dropping eigenmodes with λ_k ≤ cutoff × max(diag(cov)); default 1e-10), rescale_shape_modes_to_positive (used only when covariance nuisances are enabled; default true), keep_work (keep the generated datacards under output_dir/work/; default true), lumi_unc and trigger_unc (flat lnN fractions, 0.016 and 0.03), mc_stat_threshold (autoMCStats event threshold, default 10), the theory_syst_json, pileup_syst_json, jes_syst_json, jer_syst_json, jms_syst_json, and jmr_syst_json inputs, and enabled_nuisances (empty = all; otherwise any of theory_pdf, theory_scale, theory_ps_isr, theory_ps_fsr, pileup, jes, jer, jms, jmr, lumi, trigger, mcstat, abcd_nonclosure, abcd_mcstat).
• selections/{theory_weights,pileup_syst,jes_syst,jer_syst,jms_syst,jmr_syst}/config.json: syst-script inputs (input_root, input_pattern, sample_config, per-tree bdt_root and signal_region_csv), submit_trees, submit_samples, min_region_events (10), output_dir, and for the jet systematics weight_branch. The removed input_root_up / input_root_down keys are rejected.

Step-by-step file flow

• mode=1 writes the pileup CSVs used by mode=0 for MC samples.
• mode=0 with the ttbar-CR config writes the ttbar_cr trees that selections/jms_jmr/jms_jmr_fit.py fits into selections/jms_jmr/output/jms_jmr_results.json, the JMS/JMR input of the signal-region conversion.
• mode=0 writes the five converted analysis trees used directly by mode=4, and as the input source for mode=6 or mode=2; the MC files also hold the <tree>__<variation> trees read by mode=11–mode=14.
• mode=6 can rewrite those converted MC trees into {signal_mixed|bkg_mixed} directories under the same dataset root with the same chunk layout using a deterministic block shuffle, which the checked-in model-training config uses by default.
• mode=2 writes the model, copied configs, and saved test-set prediction references used by mode=3, mode=4, and mode=5.
• mode=3 writes signal_region.csv, which defines the A-region score bins for mode=5.
• mode=5 writes the ABCD validation ROOT file and PDFs for the chosen tree.
• mode=4 writes one physical-normalization data/MC comparison PDF per branch plus a _normal.pdf unit-area version.
• mode=7 reads one or more mode=5 ROOT files (one per channel) plus the BDT and sample configs, generates per-scenario datacards, and calls combine to fill significance.csv, limits.csv, significance_abcd_mc.csv, and limits_abcd_mc.csv under its output_dir; it also reruns the same scenarios per individual channel and writes the matching *_by_channel*.csv files. Combined/per-class scenarios use the stored groups/*/srN/yield one-bin histograms, per-sample scenarios use samples/*/srN/yield, and ABCD-mode scenarios replace every QCD class with the merged qcd_predict/srN/yield. Each SR is one bin of a one-bin-per-SR shape card whose histogram errors carry the MC statistics for autoMCStats; the lnN nuisances come from the mode=8, 9, 11–14 JSON files, the flat lumi/trigger values, and the ABCD scale factors in metadata/abcd_closure. Setting use_root_covariance=true (with mcstat disabled) adds the optional eigen-decomposed covariance shape nuisances.
• mode=8, mode=9, and mode=11–mode=14 read the converted trees, the trained model, and signal_region.csv, and write the *_syst_yields.json ratios read by mode=7.

Jet energy corrections

convert_branch.C (JetPtCorrector) corrects the AK4 ScoutingPFJetRecluster2 and AK8 ScoutingFatPFJetRecluster jets (plain PF jets, neither CHS nor PUPPI, matching AK4PFHLT/AK8PFHLT) and the MET before any selection or output reads an event. The jet_pt_correction block of the convert config configures it.

• JEC (MC and data): the official HLT MC-truth JEC Winter24HLT_V1_MC_L1L2L3Res_{AK4,AK8}PFHLT (JECDatabase); there is no residual (data/MC) correction (L2L3Residual returns 1.0), and the JERC group recommends these JECs for the 2024 simulation and the 2024CDEFGHI data. The JSON files are copied from /eos/cms/store/group/phys_jetmet/ExpJEC/json_files_HLT/Run3Winter24/ into selections/convert/jec/Run3Winter24/. The AK4 inputs already carry this JEC from the ScoutingNano production (raw = stored × (1 − rawFactor) for pT and mass), so the stored AK4 pT and mass are the nominal JEC result; the AK8 inputs are stored raw and get the JEC.
• Event rho: the inputs do not store rho, so it is recovered from the AK4 production JEC: stored / raw = C(JetA, JetEta, JetPhi, raw pT, rho) falls with rho up to the L1FastJet clamp at 70 GeV, one jet gives rho, and every AK4 jet must reproduce its stored factor within 0.5 GeV of it (a jet whose stored eta or phi lies on a JEC bin edge may use the neighbouring bin). An event whose AK4 jets have no common rho stops the conversion with an error naming the event. On the v4 data, which store ScoutingRho_fixedGridRhoFastjetAll, recovered − true rho has a mean of −0.004 GeV and an RMS of 0.018 GeV, without failure in 20000 events.
• JER (MC): the offline proxy Summer23BPixPrompt23_RunD_JRV1_MC_{PtResolution,ScaleFactor}_AK4PFPuppi (JME-POG 2024_Summer24/jet_jerc.json.gz) with the stochastic JERSmear tool (jer_smear.json.gz, GenPt = −1), for AK4 and AK8 alike: CMS provides no JER for scouting jets and none for AK8. The nominal MC is smeared with the nom scale factor. The resolution lookup clamps rho below 52.05, the end of its Rho axis. JERSmear draws its random number from (JetPt, JetEta, Rho, EventID), which are the same in every configuration, so jer_up / jer_down change only the scale factor. Data are not smeared.
• JES uncertainty (MC): a flat ±10% per jet (jes_shift) on pT, mass, and soft-drop mass after the JER smearing, one nuisance for AK4 and AK8 (fully correlated). Source: the JERC recommendation for scouting jets quotes the closure of the online HLT JECs on scouting jets as "within < 2% for ScoutingPFJet and within < 5-10% for ScoutingPFJetRecluster" (presentations: https://indico.cern.ch/event/1661768/, https://indico.cern.ch/event/1667359/); both our collections are reclustered, and 10% is the upper edge. The value covers only the non-closure of the MC-truth correction: residual (data/MC) corrections are neither applied nor covered, and CMS gives no closure for scouting AK8 jets, so using the same value for AK8 is our choice.
• AK8 soft-drop mass: every factor of the jet pT except L1FastJet (grooming removes most of the pileup), i.e. L2L3, the JER smearing, and the JES shift, and, for MC with apply_jms_jmr, the CMS-standard JMS scale and stochastic JMR smearing m × JMS × (1 + N(0,1) × sqrt(max(JMR² − 1, 0)) × σ_MC/μ_MC) (no smearing for JMR ≤ 1), with JMS, JMR, and the MC W-peak width σ_MC/μ_MC from selections/jms_jmr/output/jms_jmr_results.json. The checked-in fit predates these corrections (JMS = 1.071, +7% on the MC soft-drop mass; JMR = 0.52); re-measure it from the corrected ttbar control region.
• Type-1 MET: MET = raw MET − Σ (corrected − L1FastJet-only) muon-subtracted pT (muon fraction muEF) over the AK4 jets whose corrected, unsmeared, muon-subtracted pT is above 15 GeV and whose EM fraction chEmEF + neEmEF is below 0.9. The JER and JES variations move the MET through the same jets.

Single pass, variation trees. Each MC event is read once and corrected with the nominal configuration and each of the variations (jes_up, jes_down, jer_up, jer_down, jms_up, jms_down, jmr_up, jmr_down). Every configuration rebuilds the collections, re-applies the tree selections, and fills <tree>__<variation> trees in the same output file. A variation tree books the variation_branches list of its tree (the BDT features, thresholds, and decorrelation branches plus run, luminosityBlock, event, weight_pu) and evaluates only the scalar outputs those branches depend on. The variation trees are identical, event by event, to the nominal trees of a conversion with debug_nominal_configuration set to that variation (a validation setting that fills the nominal trees with one configuration). Data get the nominal configuration only.

Keys. enabled, nominal_correction (hlt_jec, or the legacy scouting_to_offline response SF of corrections_file with ak4_tag_threshold / ak8_tag_threshold), jec_ak4_file, jec_ak8_file, jec_ak4_name, jec_ak8_name, jec_ak4_l1_name, jec_ak8_l1_name, jes_jer_file, jer_smear_file, jer_resolution_name, jer_scale_factor_name, jes_shift, apply_jms_jmr, jms_jmr_results_file, variations, variation_branches, and debug_nominal_configuration. The keys variation, jes_unc_name, and jer_rho_fallback of the removed one-variation-per-conversion scheme are rejected; selections/convert/config_{jes,jer}_{up,down}.json are kept for reference only. The event preselection runs before the corrections on input scalars and must not read ScoutingMET_pt / ScoutingMET_phi. variation_branches must list, per tree, every BDT feature (selections/BDT/branch.json), threshold branch (selection.json), and decorrelation branch (config.json decorrelate) of the trained model, plus run, luminosityBlock, event, and weight_pu; after a change of the BDT inputs, update it (the jet syst scripts stop with an error naming a missing branch).

Cost. On one QCD HT1500–2000 file (about 200k preselected events) the corrections add about 12% CPU to a nominal-only conversion (7% for data), and the eight MC variation configurations bring the total to about 4.2 times the nominal-only conversion: each variation costs about 40% of a separate conversion, because the input is read once and the variation trees evaluate only their needed outputs. Runtime collections independent of the jet corrections (the GenPart-based w_quarks, wz_had_quarks) are built once per event (logged at startup). Plan the MC conversion jobs for that CPU time.

Environment. convert_branch.C builds in the CMSSW_16_1_0_pre4 (el9_amd64_gcc13) environment with CMSSW's correctionlib 2.7.0. run.py locates correctionlib through python3 -c "import correctionlib", so a user-site copy (~/.local/lib/python3.9/site-packages/correctionlib) takes precedence over CMSSW's; check python3 -c "import correctionlib; print(correctionlib.__version__)" before building.

Systematic uncertainties

combine.C fits the Asimov pseudo-data (the sum of the nominal process rates, -t -1). Every channel card contains:

Nuisance	Type, value	Processes	Correlation
lumi	lnN 1.016 (lumi_unc)	MC processes	all processes, SRs, and channels
trigger	lnN 1.03 (trigger_unc), a flat value set by the user; no trigger scale factor is measured	MC processes	all
pileup	lnN from weight_pu_up / weight_pu_down (mode 9)	MC processes	all
theory_pdf	lnN, symmetric-Hessian PDF (+ αs when stored) spread (mode 8)	MC processes; samples without theory weights count with ratio 1	all
theory_scale	lnN, 7-point μR/μF envelope (mode 8)	as theory_pdf	all
theory_ps_isr, theory_ps_fsr	lnN, parton-shower murfac 0.5 (up) and 2.0 (down) (mode 8)	as theory_pdf	all
jes	lnN, flat ±10% per jet (mode 11)	MC processes	all
jer	lnN, JER scale factor up/down (mode 12)	MC processes	all
jms	lnN, JMS ± its uncertainty (mode 13)	MC processes	all
jmr	lnN, JMR ± its uncertainty (mode 14; no effect while JMR ± σ ≤ 1)	MC processes	all
MC statistics (mcstat)	* autoMCStats 10 0 1 from the bin errors sqrt(sum w^2) of the test split (include-signal = 0 still models the signal statistics)	all processes; for bkg_qcd_abcd the per-SR part	per bin
abcd_nonclosure_<channel>	lnN 1 + |k − 1|, k = final_scale = pred_union / A_union	bkg_qcd_abcd	all SRs of the channel; independent between channels
abcd_mcstat_<channel>	lnN 1 + σ(pred_union) / pred_union, the B/C/D MC statistics	bkg_qcd_abcd	all SRs of the channel; independent between channels

The per-sample ratios come from the syst scripts (per SR; regions with fewer than 10 MC events take the inclusive ratio). A process kappa in an SR is the yield-weighted mean of its samples' ratios, weighted by the test-split SR yields of qcd_est.py; asymmetric kappas are written κ_down/κ_up. A non-positive kappa (a variation removing the whole yield) is an error. The MC-based nuisances do not act on the ABCD prediction bkg_qcd_abcd, which in this Asimov study is built from MC B/C/D regions and takes the ABCD rows instead; in the MC-true scenarios the QCD class is an MC process with every MC nuisance.

Not included:

• Residual (data/MC) JEC and its uncertainty, and a JES split into sources: no HLT residuals exist for scouting jets.
• A scouting or AK8 JER, and the hybrid (gen-matched) JER smearing: only the offline AK4PFPuppi proxy exists, and GenJetAK8 is available only in the jetMatchFix samples.
• The unclustered-energy MET uncertainty: the Type-1 MET propagates only the jet corrections and their variations.
• Jet-tagging scale factors (the AK8 ParT W/V tagging and the AK4 b-veto): none exist for scouting jets.
• Lepton identification, isolation, and veto efficiencies.
• Cross-section (normalisation) uncertainties of the signal and background processes.
• Data statistics of the ABCD B/C/D regions: the ABCD prediction uses MC; with data, those statistics replace abcd_mcstat_<channel>.