I got 3× faster HFQ4 prefill on Strix Halo in hipfire with an opt-in MMQ path

via

LocalLlama

I recently contributed an experimental HFQ4-G256 MMQ prefill path to hipfire, an RDNA-focused LLM inference engine. Disclaimer: I authored the PR, so this is partly a contribution note, but I am mainly looking for independent validation from other AMD users. Before this PR, HFQ4 prefill in hipfire was going through a more generic/slower path. On my Strix Halo system, prompt processing was clearly the bottleneck: longer prefills were around ~310–340 tok/s. The new path adds an opt-in MMQ-style pr

More from LocalLlama

daily ritual at this point…

LocalLlama·14

convert : add support for Nemotron Nano 3 Omni by danbev · Pull Request #22481 · ggml-org/llama.cpp

LocalLlama·5

Mistral Medium Is On The Way

LocalLlama·14

Something from Mistral (Vibe) tomorrow

LocalLlama·7

Deepseek Vision Coming

LocalLlama·8

First direct side by side MoE vs Dense comparison.

LocalLlama·9