FP4 inference in llama.cpp (NVFP4) and ik_llama.cpp (MXFP4) landed - Finally

Both llama.cpp and ik_llama.cpp now have FP4 support — but with different flavors worth knowing about. llama.cpp recently merged NVFP4 (Nvidia's block-scaled FP4, `GGML_TYPE_NVFP4 = 40`), with CUDA kernels landing in `mmq.cuh`, `mmvq.cu`, `convert.cu` and others. ik_llama.cpp has had MXFP4 (`GGML_TYPE_MXFP4 = 39`) since PR #682 — the MX-standard FP4 used in gpt-oss models. Coverage is actually broader: CPU (AVX2, NEON, Zen4), CUDA, are all implemented. They're not the same wire format — NVFP4 is

More from Reddit

Trump warns Iran blockade could last months, sending oil soaring

Reddit·7

Is 24/7 customer support a requirement to stay competitive or 9-5 is still enough for a startup in 2026?

Reddit·2

For teams using AI agents in complex codebases, where does context break down most often?

Reddit·4

Thinking of dual-boot, need concrete guidance, aim is programming and dev

Reddit·4

Israel intercepts Gaza flotilla off the coast of Crete.

Reddit·5

interview small business market in the US!

Reddit·3