Skip to content

Fix OpenML checkpoint directory and filenames#46

Open
charlesmartin14 wants to merge 1 commit intomainfrom
codex/fix-notebook-data-saving-location-and-filenames
Open

Fix OpenML checkpoint directory and filenames#46
charlesmartin14 wants to merge 1 commit intomainfrom
codex/fix-notebook-data-saving-location-and-filenames

Conversation

@charlesmartin14
Copy link
Copy Markdown
Member

Motivation

  • The notebook created an extra intermediate runs folder and embedded the OpenML id in every checkpoint filename, making directories and filenames verbose and brittle.
  • The intent is to keep the dataset/model id in the folder name but use stable, id-agnostic filenames inside the folder so checkpoints are easier to locate and manage.

Description

  • Remove the intermediate RUN_NAME_BASE folder and set RUNS_ROOT = Path('/content/drive/MyDrive/xgbww_runs') so runs are stored directly under that root.
  • Change per-dataset checkpoint folder naming to openml_<dataset_id>_w1w2w7w8w9 (removed the _alpha suffix).
  • Replace per-file names that included the dataset id with generic checkpoint filenames: state.json, metrics.csv, model_latest.json, data_split.npz, summary.json, and model_meta.json.
  • Update the notebook logic that writes model_meta to reference the new runs_root, checkpoint_dir, and the new checkpoint_files names.

Testing

  • Ran python -m json.tool XGBWW_OpenML_W1W2W7W8W9_for_openml_id.ipynb to validate the notebook JSON, which succeeded.
  • Verified the updated path and filename patterns with rg searches for the new RUNS_ROOT, DRIVE_ROOT, and checkpoint path variables, which matched the expected replacements.
  • Confirmed the notebook cell that derives dataset_id still populates the folder name so random or user-specified ids remain present in the directory name.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant