Skip to content

Commit 547f5cb

Browse files
committed
feat(cli): add competitions data push command
Wrap kagglesdk 0.1.32's new create_competition_data RPC behind `kaggle competitions data push <competition> -p folder -m "notes" [--rerun] [-r {skip,zip,tar}]`. Walks the folder, uploads each file via the existing ResumableUploadContext + blob plumbing (using ApiBlobType.INBOX — see TODO; may need adjustment based on backend feedback), then sends a single CreateCompetitionData request bundling the resulting tokens. Each push replaces the prior version's file set in full. Bumps the kagglesdk floor to 0.1.32.
1 parent 073b2be commit 547f5cb

5 files changed

Lines changed: 403 additions & 4 deletions

File tree

docs/competition_creation.md

Lines changed: 58 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,9 @@ public competition-creation API endpoints (kagglesdk 0.1.31+):
77
- [`kaggle competitions create`](#kaggle-competitions-create)
88
- [`kaggle competitions pages create`](#kaggle-competitions-pages-create)
99
- [`kaggle competitions launch`](#kaggle-competitions-launch)
10+
- [`kaggle competitions data push`](#kaggle-competitions-data-push)
1011

11-
All four commands require an authenticated session
12+
All of these commands require an authenticated session
1213
(`kaggle config set username/password` or an API token).
1314

1415
A typical end-to-end host workflow looks like:
@@ -27,11 +28,14 @@ kaggle competitions create -p ./my-comp
2728
kaggle competitions pages create my-comp-slug --name description -f ./description.md --publish
2829
kaggle competitions pages create my-comp-slug --name rules -f ./rules.md --publish
2930

30-
# 5. Launch the competition (now, or schedule a future UTC time).
31+
# 5. Push the competition data (train.csv, test.csv, sample_submission.csv, ...).
32+
kaggle competitions data push my-comp-slug -p ./data -m "Initial release"
33+
34+
# 6. Launch the competition (now, or schedule a future UTC time).
3135
kaggle competitions launch my-comp-slug --at 2027-01-01T00:00:00Z
3236
```
3337

34-
The four commands are independent — for example, you can call `pages create`
38+
These commands are independent — for example, you can call `pages create`
3539
on a competition that already exists, or use `launch` on a competition created
3640
via the host wizard.
3741

@@ -245,3 +249,54 @@ kaggle competitions launch my-comp --at 2027-01-01T00:00:00Z
245249

246250
A competition can only be launched once. Subsequent calls will be rejected by
247251
the backend.
252+
253+
---
254+
255+
## `kaggle competitions data push`
256+
257+
Pushes (versions) the data files for a competition you host. Walks the
258+
supplied folder, uploads each file via the standard blob-upload pipeline, and
259+
then sends a single request bundling the uploaded tokens. Each push **replaces
260+
the prior version's file set in full** — there is no per-file "keep from
261+
previous" mode in v1, so list every file you want in the new version.
262+
263+
**Usage:**
264+
265+
```bash
266+
kaggle competitions data push <competition> -p <folder> -m "<version notes>" \
267+
[--rerun] [-r {skip,zip,tar}]
268+
```
269+
270+
**Arguments:**
271+
272+
- `<competition>`: The competition slug.
273+
274+
**Options:**
275+
276+
- `-p, --path <folder>` (required): Folder containing the files to push.
277+
- `-m, --message "<notes>"` (required): Notes describing this version
278+
(e.g. `"Added test set"`).
279+
- `--rerun` (optional): Push to the RERUN databundle — the private host-only
280+
data swapped in during rerun scoring. Requires the
281+
`CompetitionPrivateDatabundle` feature flag server-side. Without this flag,
282+
the push targets the PUBLIC databundle (what participants download).
283+
- `-r, --dir-mode {skip,zip,tar}` (optional): How to handle sub-directories.
284+
`skip` (default) ignores them; `zip` / `tar` archives each subfolder into a
285+
single upload.
286+
287+
**Examples:**
288+
289+
```bash
290+
# Push the initial public data.
291+
kaggle competitions data push my-comp -p ./data -m "Initial release"
292+
293+
# New version with a bug-fix.
294+
kaggle competitions data push my-comp -p ./data -m "Fix label encoding in train.csv"
295+
296+
# Push private rerun-scoring data (requires feature flag).
297+
kaggle competitions data push my-comp -p ./rerun-data \
298+
-m "Held-out test set" --rerun
299+
```
300+
301+
The command prints the public URL plus the new `databundle_id` and
302+
`databundle_version_id` on success.

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ keywords = ["Kaggle", "API"]
2424
requires-python = ">= 3.11"
2525
dependencies = [
2626
"bleach",
27-
"kagglesdk >= 0.1.31, < 1.0", # sync with kagglehub
27+
"kagglesdk >= 0.1.32, < 1.0", # sync with kagglehub
2828
"python-slugify",
2929
"requests",
3030
"python-dateutil",

src/kaggle/api/kaggle_api_extended.py

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,9 @@
9999
ApiListCompetitionPagesRequest,
100100
ApiListCompetitionPagesResponse,
101101
ApiCreateCompetitionPageRequest,
102+
ApiCreateCompetitionDataRequest,
103+
ApiCreateCompetitionDataResponse,
104+
ApiCompetitionDataFile,
102105
ApiCompetitionPage,
103106
ApiCreateCompetitionRequest,
104107
ApiCreateCompetitionResponse,
@@ -133,6 +136,7 @@
133136
)
134137
from kagglesdk.competitions.types.competition_enums import (
135138
CompetitionListTab,
139+
CompetitionDatabundleType,
136140
CompetitionPrivacy,
137141
HostSegment,
138142
CompetitionSortBy,
@@ -2430,6 +2434,101 @@ def competition_create_page_cli(
24302434
status = "published" if page.is_published else "staged (unpublished)"
24312435
print(f'Page "{page.name}" created on competition "{competition_name}" — {status}.')
24322436

2437+
def competition_data_push(
2438+
self,
2439+
competition_name: str,
2440+
folder: str,
2441+
version_notes: str,
2442+
rerun: bool = False,
2443+
quiet: bool = False,
2444+
dir_mode: str = "skip",
2445+
) -> ApiCreateCompetitionDataResponse:
2446+
"""Push (version) the data files for a competition you host.
2447+
2448+
Walks ``folder``, uploads each file via the blob-upload pipeline, and
2449+
sends a CreateCompetitionData request bundling the resulting tokens.
2450+
Each push replaces the prior version's file set in full.
2451+
2452+
Args:
2453+
competition_name (str): The competition name (slug).
2454+
folder (str): Folder containing the files to push.
2455+
version_notes (str): Notes describing this version (required).
2456+
rerun (bool): If True, push to the RERUN databundle (private
2457+
host-only data swapped in during rerun scoring). Requires the
2458+
CompetitionPrivateDatabundle feature flag server-side.
2459+
quiet (bool): Suppress per-file upload progress lines.
2460+
dir_mode (str): What to do with sub-directories: "skip" (default),
2461+
"zip", or "tar".
2462+
2463+
Returns:
2464+
ApiCreateCompetitionDataResponse: url, databundle_id,
2465+
databundle_version_id of the new version.
2466+
"""
2467+
if not os.path.isdir(folder):
2468+
raise ValueError("Invalid folder: " + folder)
2469+
if not version_notes or not version_notes.strip():
2470+
raise ValueError("--message/-m version notes are required")
2471+
2472+
files: List[ApiCompetitionDataFile] = []
2473+
# TODO: confirm with backend whether competition data should use
2474+
# ApiBlobType.INBOX (used here as the closest catch-all) or whether a
2475+
# dedicated COMPETITION_DATA blob type needs adding.
2476+
with ResumableUploadContext() as upload_context:
2477+
for file_name in sorted(os.listdir(folder)):
2478+
upload_file = self._upload_file_or_folder(
2479+
folder, file_name, ApiBlobType.INBOX, upload_context, dir_mode, quiet
2480+
)
2481+
if upload_file is not None:
2482+
f = ApiCompetitionDataFile()
2483+
f.name = upload_file.name
2484+
f.token = upload_file.token
2485+
files.append(f)
2486+
2487+
if not files:
2488+
raise ValueError(f"No files found in {folder} to upload")
2489+
2490+
with self.build_kaggle_client() as kaggle:
2491+
request = ApiCreateCompetitionDataRequest()
2492+
request.competition_name = competition_name
2493+
request.version_notes = version_notes
2494+
request.files = files
2495+
if rerun:
2496+
request.competition_databundle_type = CompetitionDatabundleType.COMPETITION_DATABUNDLE_TYPE_RERUN
2497+
return kaggle.competitions.competition_api_client.create_competition_data(request)
2498+
2499+
def competition_data_push_cli(
2500+
self,
2501+
competition=None,
2502+
competition_opt=None,
2503+
folder=None,
2504+
version_notes=None,
2505+
rerun=False,
2506+
quiet=False,
2507+
dir_mode="skip",
2508+
):
2509+
"""CLI wrapper for competition_data_push."""
2510+
competition_name = competition or competition_opt
2511+
if competition_name is None:
2512+
competition_name = self.get_config_value(self.CONFIG_NAME_COMPETITION)
2513+
if competition_name is not None and not quiet:
2514+
print("Using competition: " + competition_name)
2515+
if competition_name is None:
2516+
raise ValueError("No competition specified")
2517+
if not folder:
2518+
raise ValueError("-p/--path folder is required")
2519+
if not version_notes:
2520+
raise ValueError("-m/--message version notes are required")
2521+
2522+
response = self.competition_data_push(
2523+
competition_name=competition_name,
2524+
folder=folder,
2525+
version_notes=version_notes,
2526+
rerun=rerun,
2527+
quiet=quiet,
2528+
dir_mode=dir_mode,
2529+
)
2530+
print(f'New data version pushed for "{competition_name}": {response.url}')
2531+
24332532
def competition_launch(self, competition_name: str, future_time: Optional[datetime] = None) -> None:
24342533
"""Launch a competition you host, optionally at a future UTC time.
24352534

src/kaggle/cli.py

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -467,6 +467,63 @@ def parse_competitions(subparsers) -> None:
467467
parser_competitions_pages_create._action_groups.append(parser_competitions_pages_create_optional)
468468
parser_competitions_pages_create.set_defaults(func=api.competition_create_page_cli)
469469

470+
# Competitions data (group: push)
471+
parser_competitions_data = subparsers_competitions.add_parser(
472+
"data",
473+
formatter_class=argparse.RawTextHelpFormatter,
474+
help=Help.command_competitions_data,
475+
)
476+
subparsers_competitions_data = parser_competitions_data.add_subparsers(title="commands", dest="command")
477+
subparsers_competitions_data.required = True
478+
subparsers_competitions_data.choices = Help.entity_data_choices
479+
480+
# Competitions data push
481+
parser_competitions_data_push = subparsers_competitions_data.add_parser(
482+
"push",
483+
formatter_class=argparse.RawTextHelpFormatter,
484+
help=Help.command_competitions_data_push,
485+
)
486+
parser_competitions_data_push_optional = parser_competitions_data_push._action_groups.pop()
487+
parser_competitions_data_push_optional.add_argument(
488+
"competition", nargs="?", default=None, help=Help.param_competition
489+
)
490+
parser_competitions_data_push_optional.add_argument(
491+
"-c", "--competition", dest="competition_opt", required=False, help=argparse.SUPPRESS
492+
)
493+
parser_competitions_data_push_optional.add_argument(
494+
"-p",
495+
"--path",
496+
dest="folder",
497+
required=True,
498+
help="Folder containing the files to push.",
499+
)
500+
parser_competitions_data_push_optional.add_argument(
501+
"-m",
502+
"--message",
503+
dest="version_notes",
504+
required=True,
505+
help='Notes describing this version (e.g. "Added test set").',
506+
)
507+
parser_competitions_data_push_optional.add_argument(
508+
"--rerun",
509+
dest="rerun",
510+
action="store_true",
511+
help="Push to the RERUN databundle (private host-only data used during rerun scoring).",
512+
)
513+
parser_competitions_data_push_optional.add_argument(
514+
"-r",
515+
"--dir-mode",
516+
dest="dir_mode",
517+
choices=["skip", "zip", "tar"],
518+
default="skip",
519+
help='How to handle sub-directories: "skip" (default), "zip", or "tar".',
520+
)
521+
parser_competitions_data_push_optional.add_argument(
522+
"-q", "--quiet", dest="quiet", action="store_true", help=Help.param_quiet
523+
)
524+
parser_competitions_data_push._action_groups.append(parser_competitions_data_push_optional)
525+
parser_competitions_data_push.set_defaults(func=api.competition_data_push_cli)
526+
470527
# Competitions launch (publish now, or schedule for a future UTC time)
471528
parser_competitions_launch = subparsers_competitions.add_parser(
472529
"launch", formatter_class=argparse.RawTextHelpFormatter, help=Help.command_competitions_launch
@@ -2026,6 +2083,7 @@ class Help(object):
20262083
"replay",
20272084
"logs",
20282085
"pages",
2086+
"data",
20292087
"launch",
20302088
"init",
20312089
"create",
@@ -2091,6 +2149,7 @@ class Help(object):
20912149
forums_topics_choices = ["list", "show"]
20922150
entity_topics_choices = ["list", "show"]
20932151
entity_pages_choices = ["list", "create"]
2152+
entity_data_choices = ["push"]
20942153
config_choices = ["view", "set", "unset"]
20952154
auth_choices = ["login", "print-access-token", "revoke"]
20962155

@@ -2164,6 +2223,8 @@ class Help(object):
21642223
command_competitions_episode_logs = "Download agent logs for a simulation episode"
21652224
command_competitions_pages = "List pages for a competition"
21662225
command_competitions_pages_create = "Create a new page on a competition you host"
2226+
command_competitions_data = "Manage a competition's data files"
2227+
command_competitions_data_push = "Push (version) the data files for a competition you host"
21672228
command_competitions_launch = "Launch a competition you host, optionally at a future UTC time"
21682229
command_competitions_init = "Initialize folder with a competition-metadata.json template"
21692230
command_competitions_create = "Create a new competition from competition-metadata.json"

0 commit comments

Comments
 (0)