Show HN: Kairo – Fail-closed LLM inference routing from RTX 5090 measurements
Kairo is a research workbench designed to optimize LLM inference on NVIDIA RTX 5090 GPUs by measuring specific workloads and routing them based on empirical data. It employs a fail-closed policy that only promotes serving configurations, such as CUDA Graphs, when they have been verified to improve throughput for exact measured conditions. Phase 1 results demonstrate significant speedups for certain models and concurrency levels, while intentionally excluding configurations that do not meet strict correctness and performance gates.
- ▪Kairo uses a fail-closed approach to route LLM inference requests only to configurations that have been explicitly measured and verified for correctness.
- ▪Phase 1 experiments on an RTX 5090 showed that CUDA Graph replay improved NVFP4 serving throughput by up to 2.62x for specific Qwen3.8-27B workloads.
- ▪The tool defines versioned workload protocols and records cache, correctness, and timing data separately to ensure reproducibility of results.
- ▪Kairo is currently a research baseline and does not replace general-purpose inference engines like vLLM or CUTLASS.
- ▪Future Phase 2 development aims to create native Blackwell inference components based on the evidence gathered in Phase 1.
Hacker News (AI / LLM) files mainly under ai. We currently carry 4,734 of its stories.
Story provenance
Source · retrieval · rights · ranking — open for full record
inspect →
Story provenance
Attribution is not the same as permission. This drawer separates discovery metadata, excerpts, WeSearch-generated summaries, reuse status, and whether the publisher receives the visit. Nothing here claims a legal grant the publisher has not made.
Record
| Original publisher | GitHub |
| Canonical URL | https://github.com/peter941221/Kairo |
| Publication time | Mon, 14 Sep 2026 02:21:23 +0000 |
| Retrieval time | 2026-09-14T02:36:50.359Z |
| Last seen | 2026-09-14T02:36:50.359Z |
| Headline source | Publisher (no WeSearch rewrite) |
| Excerpt source | publisher body |
| Excerpt method | First ~120 words (~800 chars) of extracted publisher body, fair-use limited. |
| Summary | WeSearch · cerebras-chat (WeSearch summarizer) |
| Summary source text | contentText |
| Citation coverage | Summary is a WeSearch-generated derivative; primary citation is the original publisher URL. |
| Cluster | ISX4EWPg5Hgc · 1 stories |
| Cluster logic | Grouped by semantic title/content similarity across sources within a rolling window. Same-publisher template collisions are excluded from coverage comparison. |
| Ranking reason | Story pages are not engagement-ranked. Hub feeds use recency, with optional source-diversified chronological ordering (cap consecutive stories per source). No personalized ranking. |
| Publisher visit | Yes — open original |
| Substitutes article? | No — link-out required for full text |
Rights status (four layers)
WeSearch handling by dimension
| Indexing | May the item be indexed (stored, ranked, made findable)? | Allowed |
| Snippet | May a short excerpt of the publisher's text be shown? | Allowed |
| AI summary | May WeSearch generate its own short summary of the article? | Limited |
| Retrieval / RAG | May the content be exposed for third-party retrieval-augmented generation? | Not asserted |
| Model training | May the content be used to train AI models? | Not asserted |
| Commercial reuse | May the content be reused commercially? | Not permitted |
Basis: Derived from the published RSS/Atom feed. Contact: [email protected]. Reviewed: 2026-07-24.
Opening excerpt (first ~120 words) tap to expand
Kairo Evidence-driven Blackwell inference research: measure a workload, prove a result, and route only what was measured. Phase 1 results · Experiment protocol · Phase 1 article · Reproduce Phase 1 · Blog · Contributing · Apache-2.0 Why Kairo On a single RTX 5090, CUDA Graph replay can improve NVFP4 serving throughput by more than 2x for some workloads—and by far less for another valid workload. The useful conclusion is not “always enable Graphs.” Kairo captures the conditions behind a result, passes a correctness gate, and promotes only exact measured workload buckets into a fail-closed runtime policy. Kairo is a research workbench and the beginning of a path toward native Blackwell inference components.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at GitHub.