Kimi Linear: An Expressive, Efficient Attention Architecture
Kimi Linear is a hybrid linear attention architecture that claims to surpass full attention in various settings, including short and long contexts and reinforcement learning scaling regimes. The model introduces Kimi Delta Attention with a fine-grained gating mechanism and a specialized chunkwise algorithm that reduces computational overhead. The authors provide open‑source implementations and pretrained checkpoints, positioning Kimi Linear as a potential drop‑in replacement for traditional attention models.
- ▪Kimi Linear incorporates Kimi Delta Attention, extending Gated DeltaNet with a finer‑grained gating mechanism to better utilize limited RNN memory.
- ▪A bespoke chunkwise algorithm using a specialized Diagonal‑Plus‑Low‑Rank transition matrix reduces computation compared to the general DPLR formulation.
- ▪The pretrained 3 billion‑parameter model outperforms full Multi‑Head Latent Attention while cutting KV cache usage by up to 75% and achieving up to six‑fold decoding speed for a 1 million token context.
- ▪The authors release the KDA kernel, vLLM implementations, and both pre‑trained and instruction‑tuned model checkpoints for further research.
Hacker News (Front Page) files mainly under programming. We currently carry 686 of its stories. Top-voted stories on Hacker News.
Story provenance
Source · retrieval · rights · ranking — open for full record
inspect →
Attribution is not the same as permission. This drawer separates discovery metadata, excerpts, WeSearch-generated summaries, reuse status, and whether the publisher receives the visit. Nothing here claims a legal grant the publisher has not made.
Record
| Original publisher | arXiv.org |
| Canonical URL | https://arxiv.org/abs/2510.26692 |
| Publication time | Tue, 28 Jul 2026 10:52:30 +0000 |
| Retrieval time | 2026-07-28T13:09:52.403Z |
| Last seen | 2026-07-28T13:09:52.403Z |
| Headline source | Publisher (no WeSearch rewrite) |
| Excerpt source | publisher body |
| Excerpt method | First ~120 words (~800 chars) of extracted publisher body, fair-use limited. |
| Summary | WeSearch · cerebras-chat (WeSearch summarizer) |
| Summary source text | contentText |
| Citation coverage | Summary is a WeSearch-generated derivative; primary citation is the original publisher URL. |
| Cluster | 6B0rkM2OVH5P · 1 stories |
| Cluster logic | Grouped by semantic title/content similarity across sources within a rolling window. Same-publisher template collisions are excluded from coverage comparison. |
| Ranking reason | Story pages are not engagement-ranked. Hub feeds use recency, with optional source-diversified chronological ordering (cap consecutive stories per source). No personalized ranking. |
| Publisher visit | Yes — open original |
| Substitutes article? | No — link-out required for full text |
Rights status (four layers)
WeSearch handling by dimension
| Indexing | May the item be indexed (stored, ranked, made findable)? | Allowed |
| Snippet | May a short excerpt of the publisher's text be shown? | Allowed |
| AI summary | May WeSearch generate its own short summary of the article? | Limited |
| Retrieval / RAG | May the content be exposed for third-party retrieval-augmented generation? | Not asserted |
| Model training | May the content be used to train AI models? | Not asserted |
| Commercial reuse | May the content be reused commercially? | Not permitted |
Basis: Derived from the published RSS/Atom feed. Contact: [email protected]. Reviewed: 2026-07-24.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Computation and Language arXiv:2510.26692 (cs) [Submitted on 30 Oct 2025 (v1), last revised 1 Nov 2025 (this version, v2)] Title:Kimi Linear: An Expressive, Efficient Attention Architecture Authors:Kimi Team: Yu Zhang, Zongyu Lin, Xingcheng Yao, Jiaxi Hu, Fanqing Meng, Chengyin Liu, Xin Men, Songlin Yang, Zhiyuan Li, Wentao Li, Enzhe Lu, Weizhou Liu, Yanru Chen, Weixin Xu, Longhui Yu, Yejie Wang, Yu Fan, Longguang Zhong, Enming Yuan, Dehao Zhang, Yizhi Zhang, T.Y.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv.org.
Discussion
0 commentsMore from arXiv.org
