Got DFlash speculative decoding working on Qwen3.5-35B-A3B with an RTX 2080 SUPER 8GB

via

LocalLlama

## Got DFlash speculative decoding working on Qwen3.5-35B-A3B with an RTX 2080 SUPER 8GB I managed to get **DFlash speculative decoding** working in llama.cpp on a pretty VRAM-limited setup. This was tested with the DFlash PR: Build tested: ```text 67cb0d507 (8942) Setup: GPU: RTX 2080 SUPER 8GB Model: Qwen3.5-35B-A3B Q5_K_M Draft model: Qwen3.5-35B-A3B-DFlash Q4_K_M Backend: CUDA The main model is a 35B MoE GGUF around 24.44 GiB , so obviously it does not fit in 8GB VRAM. The trick was combinin

More from LocalLlama

Blog: AI evals are becoming the new compute bottleneck

LocalLlama·4

Qwen3.6-27B - Closed-loop SVG Images

LocalLlama·2

MiMo-V2.5-Pro - the actual best open-weights model

LocalLlama·4

Update on 2019 Mac Pro

LocalLlama·7

16x Spark Cluster (Build Update)

LocalLlama·7

daily ritual at this point…

LocalLlama·20