Skip to content

Commit 403ee0d

Browse files
corticalstackcorticalstack
authored
Merge pull request #17 from corticalstack/fix/section-15-finetune-dependency-group
fix(15): make the fine-tuning pipeline runnable end-to-end
2 parents e6c3598 + 2e0a679 commit 403ee0d

10 files changed

Lines changed: 940 additions & 163 deletions

‎.gitignore‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,4 +42,9 @@ docs/05-custom-mcp-server/maestro-mcp/data/
4242
# Ralph / tmux tooling
4343
.ralph/
4444
.tmux/
45-
.tmux.ralph.conf
45+
.tmux.ralph.conf
46+
# Section 15 fine-tuning artifacts (regenerable / too large to commit)
47+
15-fine-tune/data/
48+
15-fine-tune/models/
49+
15-fine-tune/eval_job.json
50+
15-fine-tune/job.json

‎15-fine-tune/15-00-fine-tune.md‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -65,29 +65,35 @@ APIM Gateway
6565
```
6666
GATEWAY_URL=https://<apim-name>.azure-api.net/openai
6767
CHAT_MODEL=gpt-4.1-mini
68+
ALPHA_GATEWAY_KEY=<existing apim subscription key for alpha>
6869
FINETUNE_FOUNDRY_PROJECT_ENDPOINT=https://aif-spoke-multi-{suffix}.services.ai.azure.com/api/projects/finetune-project
69-
FINETUNE_GATEWAY_KEY=<apim subscription key for finetune>
7070
FINETUNE_APIM_CONNECTION=finetune-apim-connection
7171
FINETUNE_RESOURCE_GROUP=rg-foundry-multi-{suffix}
7272
FINETUNE_STORAGE_ACCOUNT=issft{suffix}
7373
FINETUNE_ACA_ENVIRONMENT=acae-finetune-{suffix}
7474
```
7575

76+
> **No dedicated finetune APIM key.** The teacher-model calls only need a valid APIM subscription key for the shared gateway, so the notebooks reuse `ALPHA_GATEWAY_KEY` (already in `.env` from the project-spoke deployment) rather than provisioning a separate `foundry-gateway-finetune` subscription. This removes one resource and one extra env variable. If you want isolated quotas later, you can create a dedicated APIM subscription and wire its key in - the call site is straightforward to swap.
77+
7678
3. **Azure CLI logged in** - run `az login` and ensure you have Contributor access to the resource group.
7779

7880
4. **`uv` installed** - install dependencies with:
7981
```bash
80-
uv sync
82+
uv sync --group finetune
8183
```
8284

85+
This section needs heavy ML dependencies (PyTorch, Hugging Face Transformers, PEFT) that are not in the base install. They live in the `finetune` dependency group in `pyproject.toml`. Plain `uv sync` will leave `torch` / `transformers` / `peft` / `matplotlib` / `azure-storage-blob` / `azure-ai-inference` missing and the notebooks will fail with `ModuleNotFoundError`. The `--group finetune` flag pulls them in.
86+
87+
> The base install was kept lean because the fine-tuning group adds ~3 GB (PyTorch + CUDA libs). Once you've synced the group it persists in `.venv`; you don't need the flag on subsequent `uv sync` calls.
88+
8389
5. **Bicep deployed** - deploy this lab's infrastructure into the existing multi-spoke resource group:
8490
```bash
8591
az deployment group create \
8692
--resource-group rg-foundry-multi-{suffix} \
8793
--template-file 15-fine-tune/main.bicep \
8894
--parameters deployerPrincipalId=$(az ad signed-in-user show --query id -o tsv) \
8995
apimUrl=$GATEWAY_URL \
90-
apimSubscriptionKey=$FINETUNE_GATEWAY_KEY \
96+
apimSubscriptionKey=$ALPHA_GATEWAY_KEY \
9197
existingAccountName=aif-spoke-multi-{suffix}
9298
```
9399

‎15-fine-tune/15-01-data-preparation.ipynb‎

Lines changed: 406 additions & 60 deletions
Large diffs are not rendered by default.

‎15-fine-tune/15-02-fine-tune.ipynb‎

Lines changed: 82 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,40 +1,43 @@
11
{
2-
"nbformat": 4,
3-
"nbformat_minor": 5,
4-
"metadata": {
5-
"kernelspec": {
6-
"display_name": "Python 3",
7-
"language": "python",
8-
"name": "python3"
9-
},
10-
"language_info": {
11-
"name": "python",
12-
"version": "3.11.0"
13-
}
14-
},
152
"cells": [
163
{
174
"cell_type": "markdown",
18-
"id": "",
195
"metadata": {},
20-
"source": "# ACA fine-tuning job\n\nThis notebook:\n1. Reads infra configuration from `FINETUNE_*` environment variables\n2. Provisions ACA environment + storage (idempotent via `azure_infra.py`)\n3. Submits the Olive LoRA fine-tuning job to an A100 GPU on Azure Container Apps\n4. Monitors job progress until completion",
21-
"outputs": []
6+
"source": [
7+
"# ACA fine-tuning job\n",
8+
"\n",
9+
"This notebook:\n",
10+
"1. Reads infra configuration from `FINETUNE_*` environment variables\n",
11+
"2. Provisions ACA environment + storage (idempotent via `azure_infra.py`)\n",
12+
"3. Submits the Olive LoRA fine-tuning job to an A100 GPU on Azure Container Apps\n",
13+
"4. Monitors job progress until completion"
14+
]
2215
},
2316
{
2417
"cell_type": "code",
25-
"id": "",
18+
"execution_count": null,
19+
"id": "58ea672f",
2620
"metadata": {},
27-
"source": "from pathlib import Path\nfrom dotenv import load_dotenv\nrepo_root = Path.cwd().parents[0]\nload_dotenv(repo_root / '.env', override=True)",
2821
"outputs": [],
29-
"execution_count": null
22+
"source": [
23+
"from pathlib import Path\n",
24+
"from dotenv import load_dotenv\n",
25+
"repo_root = Path.cwd().parents[0]\n",
26+
"load_dotenv(repo_root / '.env', override=True)"
27+
]
3028
},
3129
{
3230
"cell_type": "code",
33-
"id": "",
31+
"execution_count": null,
32+
"id": "cd5fc620",
3433
"metadata": {},
35-
"source": "import os\nfrom azure.identity import DefaultAzureCredential\nfrom azure.storage.blob import BlobServiceClient\nfrom azure_infra import provision_infrastructure, submit_finetune_job, monitor_job",
3634
"outputs": [],
37-
"execution_count": null
35+
"source": [
36+
"import os\n",
37+
"from azure.identity import DefaultAzureCredential\n",
38+
"from azure.storage.blob import BlobServiceClient\n",
39+
"from azure_infra import provision_infrastructure, submit_finetune_job, monitor_job"
40+
]
3841
},
3942
{
4043
"cell_type": "markdown",
@@ -48,27 +51,74 @@
4851
},
4952
{
5053
"cell_type": "code",
51-
"id": "",
54+
"execution_count": null,
55+
"id": "06f37c66",
5256
"metadata": {},
53-
"source": "# Configuration from environment variables\nRESOURCE_GROUP = os.getenv(\"FINETUNE_RESOURCE_GROUP\")\nSTORAGE_ACCOUNT = os.getenv(\"FINETUNE_STORAGE_ACCOUNT\")\nACA_ENV = os.getenv(\"FINETUNE_ACA_ENVIRONMENT\")\nBASE_MODEL_ID = \"microsoft/Phi-4-mini-instruct\"\nLOCATION = \"swedencentral\" # ACA GPU NC24-A100 only available in Sweden Central\nCONTAINER_NAME = \"ft\"\nJOB_NAME = \"iss-ft-job\"\n\nprint(f\"Resource Group: {RESOURCE_GROUP}\")\nprint(f\"Storage Account: {STORAGE_ACCOUNT}\")\nprint(f\"ACA Environment: {ACA_ENV}\")\nprint(f\"Base Model: {BASE_MODEL_ID}\")",
5457
"outputs": [],
55-
"execution_count": null
58+
"source": [
59+
"# Configuration from environment variables\n",
60+
"RESOURCE_GROUP = os.getenv(\"FINETUNE_RESOURCE_GROUP\")\n",
61+
"STORAGE_ACCOUNT = os.getenv(\"FINETUNE_STORAGE_ACCOUNT\")\n",
62+
"ACA_ENV = os.getenv(\"FINETUNE_ACA_ENVIRONMENT\")\n",
63+
"BASE_MODEL_ID = \"microsoft/Phi-4-mini-instruct\"\n",
64+
"LOCATION = \"swedencentral\" # ACA GPU NC24-A100 only available in Sweden Central\n",
65+
"CONTAINER_NAME = \"finetune\" # 3-63 chars required; \"ft\" was too short and failed silently\n",
66+
"JOB_NAME = \"iss-ft-job\"\n",
67+
"\n",
68+
"print(f\"Resource Group: {RESOURCE_GROUP}\")\n",
69+
"print(f\"Storage Account: {STORAGE_ACCOUNT}\")\n",
70+
"print(f\"ACA Environment: {ACA_ENV}\")\n",
71+
"print(f\"Base Model: {BASE_MODEL_ID}\")"
72+
]
5673
},
5774
{
5875
"cell_type": "code",
59-
"id": "",
76+
"execution_count": null,
77+
"id": "4172766e",
6078
"metadata": {},
61-
"source": "print(f\"Training Config: {RESOURCE_GROUP} | {LOCATION}\")\n\n# 1. Provision Infrastructure\nenv_id = provision_infrastructure(RESOURCE_GROUP, LOCATION, STORAGE_ACCOUNT, CONTAINER_NAME, ACA_ENV, \"data/train.jsonl\")\n\n# 2. Submit Job\nsubmit_finetune_job(\n JOB_NAME, RESOURCE_GROUP, env_id, STORAGE_ACCOUNT, CONTAINER_NAME, \n base_model=BASE_MODEL_ID, location=LOCATION\n)\n\n# 3. Monitor (this takes ~15-20 mins)\nsuccess = monitor_job(JOB_NAME, RESOURCE_GROUP)\n",
6279
"outputs": [],
63-
"execution_count": null
80+
"source": [
81+
"print(f\"Training Config: {RESOURCE_GROUP} | {LOCATION}\")\n",
82+
"\n",
83+
"# 1. Provision Infrastructure\n",
84+
"env_id = provision_infrastructure(RESOURCE_GROUP, LOCATION, STORAGE_ACCOUNT, CONTAINER_NAME, ACA_ENV, \"data/train.jsonl\")\n",
85+
"\n",
86+
"# 2. Submit Job\n",
87+
"submit_finetune_job(\n",
88+
" JOB_NAME, RESOURCE_GROUP, env_id, STORAGE_ACCOUNT, CONTAINER_NAME, \n",
89+
" base_model=BASE_MODEL_ID, location=LOCATION\n",
90+
")\n",
91+
"\n",
92+
"# 3. Monitor (this takes ~15-20 mins)\n",
93+
"success = monitor_job(JOB_NAME, RESOURCE_GROUP)\n"
94+
]
6495
},
6596
{
6697
"cell_type": "code",
67-
"id": "",
98+
"execution_count": null,
99+
"id": "6057ae8d",
68100
"metadata": {},
69-
"source": "if success:\n print(\"✅ Fine-tuning completed successfully!\")\n print(\"Adapter saved to blob storage. Will evaluate on ACA next.\")\nelse:\n print(\"❌ Training failed. Check Azure Portal for logs.\")",
70101
"outputs": [],
71-
"execution_count": null
102+
"source": [
103+
"if success:\n",
104+
" print(\"✅ Fine-tuning completed successfully!\")\n",
105+
" print(\"Adapter saved to blob storage. Will evaluate on ACA next.\")\n",
106+
"else:\n",
107+
" print(\"❌ Training failed. Check Azure Portal for logs.\")"
108+
]
109+
}
110+
],
111+
"metadata": {
112+
"kernelspec": {
113+
"display_name": "Python 3",
114+
"language": "python",
115+
"name": "python3"
116+
},
117+
"language_info": {
118+
"name": "python",
119+
"version": "3.11.0"
72120
}
73-
]
121+
},
122+
"nbformat": 4,
123+
"nbformat_minor": 5
74124
}

0 commit comments

Comments
 (0)