We benchmarked gpt-oss-120b across 6 inference providers and found a 10x throughput spread

We ran a benchmark across 10+ LLM routers, providers, and inference backends to answer the questions that come up every time someone picks a provider. Key findings: Do LLM routers add latency? No, OpenRouter was actually 70ms faster than OpenAI direct on time to first token (0.640s vs 0.712s) and Opper matched OpenAI directly within confidence intervals. How much does AWS region selection matter? Geography dominates model choice, Tokyo was 2x slower than Ireland (3.08s vs 1.61s), a bigger impact

More from LocalLlama

daily ritual at this point…

LocalLlama·14

convert : add support for Nemotron Nano 3 Omni by danbev · Pull Request #22481 · ggml-org/llama.cpp

LocalLlama·5

Mistral Medium Is On The Way

LocalLlama·14

Something from Mistral (Vibe) tomorrow

LocalLlama·6

Deepseek Vision Coming

LocalLlama·7

First direct side by side MoE vs Dense comparison.

LocalLlama·8