fix: TEM-31, TEM-32, TEM-39 pip warm-up, RunPod REST API v2 in smoke tests, proxy-first checks - #30
Open
Chmokachka wants to merge 7 commits into
Open
fix: TEM-31, TEM-32, TEM-39 pip warm-up, RunPod REST API v2 in smoke tests, proxy-first checks#30Chmokachka wants to merge 7 commits into
Chmokachka wants to merge 7 commits into
Conversation
Chmokachka
marked this pull request as ready for review
August 10, 2026 09:19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes TEM-31, TEM-32, TEM-39.
Summary
Image fixes
python -m pip --versionwith a 5s timeout; on a cold container the first pip invocation could exceed it and Manager reported "Neither pip nor uv are available".start.shnow warms pip up before ComfyUI starts and logs the wall time. A matching always-on pip check in the smoke tests fails the run if pip is missing or slower than Manager's 5s budget.NVIDIA_DRIVER_CAPABILITIESfromalltocompute,utility,video. Requestinggraphics/displaymakes the container toolkit bind-mount Vulkan/GLX files (e.g./etc/vulkan/icd.d/nvidia_icd.json), which aborts container start on hosts with headless drivers (the CZ-1 RTX PRO 6000 WK incident). ComfyUI needs no OpenGL/Vulkan;videois kept for NVENC/NVDEC.Smoke tests: RunPod REST API v2 adoption
GET /v2/pods/{id}. SSH stays the readiness gate, but the v2statusis polled alongside: unlike the legacydesiredStatus(RUNNING from second one, even when the container never starts), v2 distinguishesSTARTING/RUNNINGand surfacesERROR— onERROR/EXITED/TERMINATEDwe bail immediately instead of sitting out the fullCREATE_TIMEOUT.GET /v2/pods/{id}/logs?source=system) on stall/timeout/terminal state and in every diagnostic dump. That's where image-pull and container-init failures are reported — container stdout is empty when the container never started. Filtered by the newSYS_LOG_ERROR_PATTERN(error/failed/crashed markers).crash(ed/es/ing)as an error marker (LOG_ERROR_PATTERN).Smoke tests: reliability
GET /server_download/folder_pathswas a single-shot probe, so a transient proxy 404/5xx was misclassified as "node not installed in this image" (intermittent CI failures). It now retries for up toCOMFYUI_ROUTES_TIMEOUT(default 60s) and the FAIL message includes the last HTTP code/error.Pipeline examples: