chore(local-inference): bump fork — gemma4-assistant MTP works in the fused engine#9273
Conversation
… fused engine
Follow-up to the gemma4-assistant arch bump: the fused engine (libelizainference,
the actual eliza runtime) built its MTP draft context without ctx_other, so a
gemma4-assistant drafter hard-failed there ("requires ctx_other"). Fixed in the
fork (elizaOS/llama.cpp#32) — set cp.ctx_other = ctx_tgt before init.
Validated end-to-end via the fused FFI on M4 Max Metal: the fused engine loads
the amaranus gemma4-assistant drafter + runs separate-drafter MTP (drafted 28 /
accepted 12 = 43%, coherent output, 112 tok/s). This is the runtime path the
plugin uses — so the Gemma-4 MTP drafter now works in eliza, not just llama-cli.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.
1 participant
Follow-up to the gemma4-assistant arch bump (#9268). The fused engine (
libelizainference, the real eliza runtime) built its MTP draft context withoutctx_other, so a gemma4-assistant drafter hard-failed there. Fixed in the fork (elizaOS/llama.cpp#32).Validated end-to-end via the fused FFI on M4 Max Metal: loads the amaranus gemma4-assistant drafter + runs separate-drafter MTP — drafted 28 / accepted 12 (43%), coherent output, 112 tok/s. The Gemma-4 MTP drafter now works in the eliza runtime, not just llama-cli.
🤖 Generated with Claude Code