[Doc] [ROCm]: Document the steps to run vLLM Omni on ROCm #144
Conversation
Signed-off-by: root <[email protected]> Signed-off-by: <> Signed-off-by: tjtanaa <[email protected]>
Signed-off-by: tjtanaa <[email protected]>
Signed-off-by: tjtanaa <[email protected]>
Signed-off-by: tjtanaa <[email protected]>
Signed-off-by: tjtanaa <[email protected]>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| ??? abstract "meta.lst" | ||
| ``````lst | ||
| --8<-- "examples/offline_inference/qwen2_5_omni/meta.lst" | ||
| `````` |
There was a problem hiding this comment.
The Qwen2.5 offline example page now pulls in meta.lst and a series of output_audio/top*.txt files, but none of those artifacts exist under examples/offline_inference/qwen2_5_omni (find examples/offline_inference/qwen2_5_omni -maxdepth 2 -type f returns only the scripts). The --8<-- snippet plugin fails hard when a target file is missing, so MkDocs builds (and this page in the rendered docs) will error until the references are removed or the files are added.
Useful? React with 👍 / 👎.
| ??? abstract "output_audio/00000.txt" | ||
| ``````txt | ||
| --8<-- "examples/offline_inference/qwen3_omni/output_audio/00000.txt" |
There was a problem hiding this comment.
This page tries to include examples/offline_inference/qwen3_omni/output_audio/00000.txt, but that file (and the output_audio directory) is not present anywhere in the repository (find examples/offline_inference/qwen3_omni -maxdepth 2 -type f lists only the scripts). The --8<-- snippet directive will throw a file-not-found error during documentation build/rendering, so the page cannot be generated as written.
Useful? React with 👍 / 👎.
Signed-off-by: tjtanaa <[email protected]>
Gaohan123
left a comment
Gaohan123
left a comment
There was a problem hiding this comment.
I think we should move hardware specific commands out of examples readme. Maybe we can create a section in docs for hardware specific commands for online and offline inference. Otherwise the readme for examples will become longer and longer.
|
What if I add those flags as a note in the README.md , e.g. on ROCm, please What do you think about the |
Signed-off-by: tjtanaa <[email protected]>
Signed-off-by: tjtanaa <[email protected]>
Signed-off-by: tjtanaa <[email protected]>
Signed-off-by: tjtanaa <[email protected]>
Signed-off-by: tjtanaa <[email protected]>
|
@Gaohan123 I have removed most of the AMD commands in the documentation and added the instruction as a NOTES. |
2 participants
PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.
Purpose
This PR documents how to setup and run vllm-omni on ROCm.
Add example instructions that works on ROCm.
Test Plan
Validated all the script locally.
Test Result
All the offline and online examples can be run.
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)