Body:
Problem
Currently, if MODEL_PATH is not set, start.sh warns and proceeds — the container
then downloads the GGUF from HuggingFace on every cold start. A Q4_K_XL GGUF for
Qwen3.5-4B is ~3 GB. This is slow, uses bandwidth, and the download is lost when
the container is removed.
Expected behaviour
The model GGUF should live on the host and be passed in via MODEL_PATH:
MODEL_PATH=/home/user/models/qwen3.5-4b-q4_k_xl.gguf \
MS_PATH=/data/3C129_1.ms \
./start.sh
start.sh mounts it read-only at /models/model.gguf inside the container.
Proposed change
- Make MODEL_PATH required in start.sh (fail with a clear error if unset).
- Document the one-time model download step in README:
one-time: download the model to your host
huggingface-cli download unsloth/Qwen3.5-4B-GGUF \
Qwen3.5-4B-UD-Q4_K_XL.gguf \
--local-dir ~/models/
- Remove HuggingFace download logic from the entrypoint/container entirely.
Why
- Container storage is ephemeral — downloaded models are lost on ./stop.sh.
- Re-downloading 3 GB on every fresh start is unacceptable for dev iteration.
- Volume-mounting keeps the model on the host where it persists across rebuilds and restarts.
Body:
Problem
Currently, if MODEL_PATH is not set, start.sh warns and proceeds — the container
then downloads the GGUF from HuggingFace on every cold start. A Q4_K_XL GGUF for
Qwen3.5-4B is ~3 GB. This is slow, uses bandwidth, and the download is lost when
the container is removed.
Expected behaviour
The model GGUF should live on the host and be passed in via MODEL_PATH:
MODEL_PATH=/home/user/models/qwen3.5-4b-q4_k_xl.gguf \
MS_PATH=/data/3C129_1.ms \
./start.sh
start.sh mounts it read-only at /models/model.gguf inside the container.
Proposed change
one-time: download the model to your host
huggingface-cli download unsloth/Qwen3.5-4B-GGUF \
Qwen3.5-4B-UD-Q4_K_XL.gguf \
--local-dir ~/models/
Why