Update PLaMo 2 Translate serve command#595
Conversation
Adjust PLaMo 2 Translate to the provided trust-remote-code and TP=4 launch command. Signed-off-by: haic0 <haichzha@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Code Review
This pull request updates the configuration and guide for the plamo-2-translate model to default to a tensor parallelism of 4. However, the reviewer notes that hardcoding tp: 4 in strategy_overrides breaks dynamic tensor parallelism scaling for single-GPU setups and recommends removing this override to allow the TP size to adapt dynamically to the selected hardware.
| strategy_overrides: | ||
| single_node_tp: | ||
| tp: 4 |
There was a problem hiding this comment.
Hardcoding tp: 4 in strategy_overrides forces the interactive CommandBuilder to always generate --tensor-parallel-size 4, even when a user selects a single-GPU hardware profile (such as an RTX 4090 or L40S) which is explicitly supported according to the guide. This breaks the dynamic tensor parallelism scaling of the single_node_tp strategy (which normally scales TP to match the selected hardware's GPU count). Removing this override allows the TP size to dynamically adapt to the selected hardware profile.
strategy_overrides: {}
Summary
Test plan
node scripts/build-recipes-api.mjson the complete validated recipe update set.Made with Cursor