chore(local-inference): non-Apple kokoro fast path — gitlink 114eee08e→2bdcef890 (Android/Windows)#11584
Conversation
…e non-Apple kokoro fast path Adds Android/Windows kokoro acceleration on top of the Apple Accelerate path already on develop (114eee08e). The non-Apple branch of the four hot vocoder primitives (Conv1d/ConvTranspose1d/Linear/LSTM gates) now uses an internal std::thread pool + NEON vfmaq_f32 on aarch64 (auto-vectorizable scalar on x86); pure scalar loop retained as reference. Apple path unchanged. Verified on real arm64: 23/23 parity checks vs scalar (max |Δ| 1.9e-5), NDK r29 cross-compile clean (18 fmla.4s in aarch64 object), microbench up to 107x on conv1d. Fork: elizaOS/llama.cpp#38 (perf/kokoro-accelerate-blas @ 2bdcef890). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
1 participant
Follow-up to #11517. That PR landed the Apple Accelerate kokoro path (submodule
114eee08e); this bumps the gitlink to2bdcef890to add the portable non-Apple fast path (Android/Windows) that landed after the merge.std::threadpool + NEONvfmaq_f32on aarch64 (auto-vectorizable scalar on x86). Apple Accelerate path unchanged; pure scalar loop retained as reference.fmla.4sin the aarch64 object), microbench up to 107× on conv1d.perf/kokoro-accelerate-blas@2bdcef890).Additive submodule bump only — no Milady-side source change. Evidence:
.github/issue-evidence/kokoro-metal-perf/.🤖 Generated with Claude Code