Simple to use vLLM Docker Container for Qwen3.6 27b with Lorbus AutoRound INT4 quant and MTP speculative decoding - 118 tokens/second on 2x 3090s
·
0 reactions
·
0 comments
·
5 views
Original article
LocalLlama
Anonymous · no account needed
Discussion
0 commentsMore from LocalLlama
daily ritual at this point…
LocalLlama·14
convert : add support for Nemotron Nano 3 Omni by danbev · Pull Request #22481 · ggml-org/llama.cpp
LocalLlama·5
Mistral Medium Is On The Way
LocalLlama·14
Something from Mistral (Vibe) tomorrow
LocalLlama·6
Deepseek Vision Coming
LocalLlama·7
First direct side by side MoE vs Dense comparison.
LocalLlama·8