First lazy-init request blocks the entire API server (health checks included) until model loading finishes

[glb99]

Did you check DeepWiki?

• No, this is a new bug not covered in the docs

Describe the bug
When the REST API server is started with lazy model loading (ACESTEP_INIT_SERVICE=false, the default), the first request that triggers model initialization blocks the entire process — including GET /health and POST /query_result for unrelated jobs — for the full duration of model loading (DiT model load, LM tokenizer load, constrained-decoding setup, and especially vLLM's torch.compile warm-up on first LM use). In our case this exceeded 90+ seconds and made the server appear completely unreachable to a downstream client, even though it was working correctly the whole time and the job eventually succeeded.

Root cause: in acestep/api/job_runtime_state.py, ensure_models_initialized calls do_model_initialization(...) as a plain synchronous call inside async def, with no await/run_in_executor/asyncio.to_thread. Since this coroutine runs on the single-threaded asyncio event loop, that call blocks the whole event loop — and therefore every other in-flight request — until it returns.

async def ensure_models_initialized(app_state: Any) -> None:
    ...
    async with app_state._init_lock:
        ...
        do_model_initialization(app=_AppProxy(app_state), **init_kwargs)  # <- blocks the event loop

By contrast, the actual generation step in acestep/api/job_execution_runtime.py gets this right, using the same ThreadPoolExecutor already sitting on app_state:

loop = asyncio.get_running_loop()
result = await loop.run_in_executor(executor, _blocking_generate)

To Reproduce
Steps to reproduce the behavior:

1. Start acestep-api (or the Docker image) with lazy init left at its default (ACESTEP_INIT_SERVICE=false).
2. Submit a POST /release_task request with thinking: true on a fresh process (nothing loaded yet).
3. Immediately poll GET /health or POST /query_result from a second client while the first request is being processed.
4. Observe: these calls do not return until model initialization (DiT + LM + vLLM compile) finishes — often 60–120+ seconds on an 8GB-tier GPU — instead of returning promptly with a "still loading"/"queued" status.

Expected behavior
GET /health and POST /query_result should stay responsive during model initialization, the same way they already do once a job is running. A client should be able to tell "still initializing" apart from "server unreachable" without an indefinite/unbounded wait on a single HTTP call.

Proposed fix
Wrap the do_model_initialization(...) call in ensure_models_initialized with the same loop.run_in_executor(executor, ...) pattern already used for generation. Flagging this as an issue rather than jumping straight to a PR since it touches a code path shared across CUDA/CPU/MPS/XPU — want to check whether there's a reason it's currently synchronous (e.g. a backend/thread-safety constraint specific to model init) before assuming run_in_executor is safe everywhere. Happy to submit the PR once that's confirmed.

Screenshots
N/A — this is a backend/API timing issue, not a UI issue.

Desktop (please complete the following information):

• OS: Windows 11 (Docker Desktop, WSL2 backend)
• Deployment: ghcr.io/ace-step/ace-step-1.5:latest (Docker), CUDA backend
• GPU: RTX 4060 Laptop, 8GB (tier3), acestep-v15-turbo + acestep-5Hz-lm-0.6B

Smartphone (please complete the following information):
N/A

Additional context
Confirmed via process inspection during the hang that the worker process stayed in a normal running state the whole time (not deadlocked/crashed) — this is a scheduling/blocking issue, not a crash. torch._inductor compile-worker subprocesses were actively spinning during the slow window, consistent with vLLM's first-use torch.compile cost. Not yet reproduced/checked on CPU/MPS/XPU backends.

[coderabbitai]

🔗 Related PRs

#671 - fix(generation): add timeout, progress fallback, and VRAM pre-flight … [merged]
#707 - fix: respect ACESTEP_INIT_LLM=false in lazy load and prioritize expli… [merged]
#741 - refactor(api): decompose api_server into focused modules [merged]
#745 - refactor(api): extract lifespan runtime and worker orchestration [merged]
#977 - feat: slot-aware DiT initialization for multi-handler API [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[ChuxiJ]

Thanks. I agree with the root cause: ensure_models_initialized() is async, but the first lazy initialization calls do_model_initialization(...) synchronously while holding the init lock, so it blocks the event loop and makes unrelated endpoints such as /health appear unavailable.

The fix should stay narrow:

• keep the existing _init_lock / double-check / _init_error semantics;
• run do_model_initialization(...) via the existing executor or asyncio.to_thread, not directly on the event loop;
• add an async regression test proving the init path awaits executor work rather than executing the blocking callable inline.

#1284 is related but covers request-time model switching/on-demand loading; it does not fully cover this first lazy-init path, so this should be tracked separately.

[glb99]

Thanks for confirming the root cause, @ChuxiJ! :) I'm working on a PR for this now — will link it here once it's up.

[glb99]

PR is up: #1296 — kept it to the scope you outlined (executor offload via the existing app_state.executor, lock/error semantics untouched, plus a deterministic regression test).

[ChuxiJ]

Thanks @glb99, this looks aligned with the scoped fix we discussed. I will move the detailed review to #1296. From the issue side, this can close once that PR lands.