fix(ci): fix vLLM emulator entrypoint and CPU runner arg in monitorin… - #107
Open
sbates130272 wants to merge 4 commits into
Open
fix(ci): fix vLLM emulator entrypoint and CPU runner arg in monitorin…#107sbates130272 wants to merge 4 commits into
sbates130272 wants to merge 4 commits into
Conversation
…g smoke Two fixes for the CPU monitoring smoke test: 1. Add `--entrypoint python3` to the vLLM emulator `docker run` in spur-monitoring-cpu-smoke.sh. The rocm-aic image ENTRYPOINT is `python3 -m vllm.entrypoints.openai.api_server`; without overriding it the command argument is appended rather than replacing it, so vLLM's API server runs instead of our emulator script and crashes on CPU-only nodes with "Failed to infer device type". 2. Fix the LLM fallback in vllm_emulator_server.py: the installed vLLM build (0.25.0 AMD fork) uses `runner="cpu"` not `device="cpu"` in EngineArgs/LLM.__init__. The old kwarg raised TypeError and aborted the emulator before it could serve any requests. Co-Authored-By: Claude <noreply@anthropic.com>
The `find` glob matched exporter tarballs (aic-nvme-exporter-*.tar.zst, aic-rdma-exporter-*.tar.zst) before the main image tarball, causing `docker load` to load the wrong image and the subsequent `docker run` for the vLLM emulator to fail with "image not found". Prefix the glob with the sanitized AIC_IMAGE name (same convention run-build-distribute.sh uses: '/:' -> '--') so only the main image tarball is matched. Co-Authored-By: Claude <noreply@anthropic.com>
…odes The rocm-aic image is a GPU-only vLLM build; platform detection requires a physical GPU and cannot be overridden via env vars. Both LLMEmulator and LLM(runner="cpu") fail at engine config creation with "Device string must not be empty". Replace the vLLM LLM fallback with a pure-Python stub that returns canned completions when LLMEmulator is unavailable. The stub emits the same Prometheus metrics and returns non-empty text for every prompt, so the CPU smoke test health checks pass without requiring GPU hardware. Co-Authored-By: Claude <noreply@anthropic.com>
Compute nodes can have a full /tmp (tmpfs or small local disk), causing `mkdir /tmp/aic-prom-tsdb-*` to fail with ENOSPC. Move METRICS_DIR under METRICS_PAGE_DIR which is already on shared NFS, so disk space is not a concern and the existing EXIT trap cleanup covers it automatically. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…g smoke
Two fixes for the CPU monitoring smoke test:
Add
--entrypoint python3to the vLLM emulatordocker runin spur-monitoring-cpu-smoke.sh. The rocm-aic image ENTRYPOINT ispython3 -m vllm.entrypoints.openai.api_server; without overriding it the command argument is appended rather than replacing it, so vLLM's API server runs instead of our emulator script and crashes on CPU-only nodes with "Failed to infer device type".Fix the LLM fallback in vllm_emulator_server.py: the installed vLLM build (0.25.0 AMD fork) uses
runner="cpu"notdevice="cpu"in EngineArgs/LLM.init. The old kwarg raised TypeError and aborted the emulator before it could serve any requests.