Summary
Two separate bugs in the model-unload endpoints:
-
POST /unload_model requires a JSON body with key model_name. Every other write endpoint in the same API (/api/generate, /api/chat, /load_model) uses model as the key. Calling /unload_model with {"model": "..."} (the natural, consistent choice given the rest of the API) returns a generic 400 with no indication of the actual expected field name — you have to read server.py to discover it wants model_name.
-
POST /unload_models (plural — intended to unload everything) doesn't actually unload anything. It returns 200 OK with {"message": "Models successfully unloaded!"} regardless, but no models are actually unloaded.
Environment
- rkllama version: 0.0.75
- Board: Radxa ROCK 5B+ (RK3588), Armbian 26.8.1
Bug 1: wrong/undocumented key name
Steps to reproduce
# Natural guess based on the rest of the API (model / load_model use "model")
curl -s -X POST http://localhost:8080/unload_model -H "Content-Type: application/json" -d '{"model":"<loaded-model-name>"}'
Actual
{"error":"Please enter the name of the model to be unloaded."}
Expected
Either the endpoint should accept model for consistency with the rest of the API, or — at minimum — the error message should name the actually-required key (model_name) instead of a generic "please enter the name" with no field-name hint.
Root cause (from source)
def unload_model_route():
data = request.get_json(force=True)
model_name = data.get('model_name', None)
if model_name is None:
return jsonify({"error": "Please enter the name of the model to be unloaded."}), 400
Bug 2: /unload_models (plural) doesn't call stop_all — silent no-op
Steps to reproduce
curl -s -X POST http://localhost:8080/unload_models
Actual
Returns 200 OK, {"message": "Models successfully unloaded!"}, but any previously-loaded model remains loaded and resident in memory (verified via free -h and GET /models / server process RSS before and after — no memory was released).
Root cause (from source)
@app.route('/unload_models', methods=['POST'])
def unload_models_route():
variables.worker_manager_rkllm.stop_all
return jsonify({"message": "Models successfully unloaded!"}), 200
variables.worker_manager_rkllm.stop_all is a bare attribute/method reference — it's never actually called (missing ()). Should be variables.worker_manager_rkllm.stop_all().
Expected behavior
/unload_model should accept model (matching every other endpoint) or clearly document/error on the actual required key.
/unload_models should actually invoke stop_all() and unload every loaded model, matching its documented/implied behavior and its success response.
Additional context
Full writeup with the underlying benchmark data (this is called out as Finding 5): https://claude.ai/code/artifact/09b27c30-c837-4204-8941-1d107ee75c9d
Summary
Two separate bugs in the model-unload endpoints:
POST /unload_modelrequires a JSON body with keymodel_name. Every other write endpoint in the same API (/api/generate,/api/chat,/load_model) usesmodelas the key. Calling/unload_modelwith{"model": "..."}(the natural, consistent choice given the rest of the API) returns a generic 400 with no indication of the actual expected field name — you have to readserver.pyto discover it wantsmodel_name.POST /unload_models(plural — intended to unload everything) doesn't actually unload anything. It returns200 OKwith{"message": "Models successfully unloaded!"}regardless, but no models are actually unloaded.Environment
Bug 1: wrong/undocumented key name
Steps to reproduce
Actual
{"error":"Please enter the name of the model to be unloaded."}Expected
Either the endpoint should accept
modelfor consistency with the rest of the API, or — at minimum — the error message should name the actually-required key (model_name) instead of a generic "please enter the name" with no field-name hint.Root cause (from source)
Bug 2: /unload_models (plural) doesn't call stop_all — silent no-op
Steps to reproduce
Actual
Returns
200 OK,{"message": "Models successfully unloaded!"}, but any previously-loaded model remains loaded and resident in memory (verified viafree -handGET /models/ server process RSS before and after — no memory was released).Root cause (from source)
variables.worker_manager_rkllm.stop_allis a bare attribute/method reference — it's never actually called (missing()). Should bevariables.worker_manager_rkllm.stop_all().Expected behavior
/unload_modelshould acceptmodel(matching every other endpoint) or clearly document/error on the actual required key./unload_modelsshould actually invokestop_all()and unload every loaded model, matching its documented/implied behavior and its success response.Additional context
Full writeup with the underlying benchmark data (this is called out as Finding 5): https://claude.ai/code/artifact/09b27c30-c837-4204-8941-1d107ee75c9d