+The "DatasetPreparator" [@Bialecki_2022_SC2DatasetPreparator] submodule is a set of scripts that ease the process of working with major collections of raw data (replaypacks/datasets). A full list of scripts is as follows: (1) "directory_flattener.py"; flattens the nested directory structure of the replaypacks, (2) "directory_packager.py"; packages all of the directories in the specified input directory, (3) "file_renamer.py"; renames the files in the directory to follow a specific naming convention (e.g., to match the dataset schema), (4) "json_merger.py"; merges two JSON files into one, (5) "processed_mapping_copier.py"; copies the auxiliary files generated by directory_flattener.py to matching output directories. Built specifically to prepare the SC2EGSet prior to packaging, (6) "sc2_map_downloader.py"; wraps "SC2InfoExtractorGo" to run the map downloading step, (7) "sc2egset_replaypack_processor.py"; wraps "SC2InfoExtractorGo" to run the replaypack processing step on multiple directories at once, (8) "sc2egset_pipeline.py"; wraps the entire processing pipeline used to obtain the "SC2ReSet" and "SC2EGSet" datasets, (9) "sc2reset_replaypack_downloader.py"; downloads the raw (flattened) replaypacks of "SC2ReSet" [@Bialecki2022ReSetZenodo] for users that wish to use their own tools for data processing.
0 commit comments