[feat][SFT] Add native Fireworks training backend

[bharatmekala]

What does this PR do?

Adds native supervised fine-tuning on dedicated Fireworks trainers while reusing the existing SFTTrainer loop.

• runs trainer-only Fireworks SFT without Ray or a rollout deployment
• reuses native data loading, batching, callbacks, logging, checkpoint cadence, and stateful dataloader resume
• saves resumable DCP state and optionally saves a promotable inference snapshot at every checkpoint cadence
• records the exact Fireworks control-plane DCP row after save, even when Fireworks renames the requested checkpoint
• serializes DCP save and identity resolution against a pre-save control-plane baseline so cross-job resume cannot select an unrelated row
• fails closed when a cross-job manifest lacks a resolved DCP identity
• reuses an existing promotable final checkpoint and deletes the trainer only after successful final promotion

Checkpoint contract

• resumable=True corresponds to DCP weights and optimizer state.
• promotable=True corresponds to an inference-formatted sampler checkpoint.
• save_promotable_checkpoints=false preserves periodic DCP-only behavior.
• DCP and sampler identities are persisted separately.
• A deleted trainer retains checkpoints for the Fireworks 30-day retention window; cross-job resume uses the exact stored DCP row.

Validation

• 148 hosted Fireworks backend and SFT config/trainer tests
• repository pre-commit hooks pass
• paid Qwen3-4B same-trainer resume and optimizer restore
• paid Qwen3-4B LoRA promotion
• paid Qwen3.5 35B A3B full-FT promotion
• paid Qwen3.6 35B A3B full-FT smoke completed 38 steps with periodic checkpoints

CI note

The fireworks base branch has repository-wide latest-Ruff drift in the separate SkyRL-CPU workflow, and fork GPU jobs lack Anyscale credentials. The scoped code-quality job passes and the hosted Fireworks suite passes locally.

Generated with Devin

[gemini-code-assist]

Code Review

This pull request introduces support for hosted Fireworks Supervised Fine-Tuning (SFT) training, allowing SkyRL to run its native SFT loop on a dedicated Fireworks trainer without requiring local GPUs or Ray workers. It adds a new direct entrypoint (main_fireworks_sft.py), SFT batch translation to Fireworks cross-entropy requests, configuration validation, and cleanup utilities. Feedback on the changes suggests optimizing the contiguous left padding check in the SFT batch processing to avoid multiple list allocations.

[gemini-code-assist]

The check for contiguous left padding can be made more efficient. The current implementation creates a new list from attention_mask, then iterates it with sum(), then creates two more lists and concatenates them for the comparison. This can be slow for long sequences in a performance-sensitive data loading path.

A more efficient approach is to iterate through the boolean mask once to validate its structure, avoiding multiple list allocations.

Suggested change

	present = [bool(value) for value in attention_mask.tolist()]
	count = sum(present)
	if present != [False] * (len(present) - count) + [True] * count:
	raise ValueError(f"attention_mask[{row_index}] must describe contiguous left padding")
	return [int(token) for token in sequences[attention_mask.bool()].tolist()]
	present = [bool(value) for value in attention_mask.tolist()]
	# Check for contiguous left padding efficiently.
	# The mask should be of the form [False, ..., False, True, ..., True].
	# It can only transition from False to True once.
	seen_true = False
	is_valid = True
	for is_present in present:
	if seen_true and not is_present:
	is_valid = False
	break
	if is_present:
	seen_true = True
	if not is_valid:
	raise ValueError(f"attention_mask[{row_index}] must describe contiguous left padding")
	return [int(token) for token in sequences[attention_mask.bool()].tolist()]

[bharatmekala]

Fixed in 284eb02. The converter now checks for a True-to-False tensor transition and reuses the boolean tensor for token extraction; the existing non-contiguous-padding regression test still passes.

[bharatmekala]

CI note: the current failures occur before this PRs feature tests run:

• SkyRL-CPU reports 2,641 repository-wide Ruff findings already present on the fireworks base branch.
• SkyRL-Train-CPU Black reformats eight pre-existing Fireworks/base files not changed by this PR.
• SkyRL-GPU cannot submit from the fork because the Anyscale token is unavailable.

Focused Fireworks/SFT tests, the existing SFT regression subset, formatting on every changed Python file, and the docs production build pass locally; exact commands and live validation are recorded in the PR body.

[cursor]

VLM models no longer fail closed

Medium Severity

The new setup() path sets is_vlm to False instead of detecting vision models, so the existing _init_workers rejection never runs. A VLM model.path now proceeds as text-only SFT, which can tokenize multimodal data without a processor and send incorrect sequences to a paid Fireworks trainer.

Additional Locations (1)

• skyrl/train/fireworks_sft_trainer.py#L37-L39

 

^{Reviewed by Cursor Bugbot for commit 2b34fb9. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{Reviewed by Cursor Bugbot for commit 9dd3ab5. Configure here.}

[cursor]

Checkpoint env vars skip trainer preserve

Medium Severity

The smoke script only forces CKPT_PATH when PRESERVE_TRAINER_FOR_RESUME=1, but cleanup_on_exit stays true whenever preserve is 0. Setting CKPT_PATH, CKPT_INTERVAL, or RESUME_FROM without preserve still prints a paid-run plan, then validate_fireworks_sft_cfg rejects the run because checkpointing now requires cleanup_on_exit=false. The confirmation text also still says the trainer will be deleted even when preserve is enabled.

Additional Locations (1)

• examples/train/sft/run_sft_fireworks.sh#L87-L91

 

^{Reviewed by Cursor Bugbot for commit 9dd3ab5. Configure here.}

[bharatmekala]

Paid deleted-trainer DCP resume validation passed on Qwen3-4B LoRA:

• source trainer: skyrl-smoke-dcp-source-a4825fc411
• source forward/backward + optimizer step succeeded
• DCP resolved to exact row .../checkpoints/step-1 with type CHECKPOINT_TYPE_TRAINING_LORA
• source trainer deleted and reached JOB_STATE_DELETED
• new target trainer: skyrl-smoke-dcp-target-a4825fc411
• target loaded cross_job://skyrl-smoke-dcp-source-a4825fc411/step-1 with load_state_with_optimizer
• post-restore forward/backward + optimizer step succeeded
• source and target trainers both verified JOB_STATE_DELETED in final cleanup

No deployment was created.