UK AISI / Caisi Preliminary Assessment of Kimi K3's Cyber Capabilities
Detailed ResultsThese results represent preliminary evaluations on a small set of public and private benchmarks. U.S. closed-weight models were evaluated with system-level safeguards disabled to reduce refusals and enable measurement of maximal capabilities. Publicly available versions of these models have these safeguards enabled.
- ▪Detailed ResultsThese results represent preliminary evaluations on a small set of public and private benchmarks.
- ▪U.S. closed-weight models were evaluated with system-level safeguards disabled to reduce refusals and enable measurement of maximal capabilities.
- ▪Publicly available versions of these models have these safeguards enabled.
2 outlets in our directory ran this story, first to last over 6 hours. All of the coverage we found sits in one bucket: centre. That one-sidedness is itself worth noticing.
Hacker News (Front Page) files mainly under programming. We currently carry 558 of its stories. Top-voted stories on Hacker News.
Opening excerpt (first ~120 words) tap to expand
Detailed ResultsThese results represent preliminary evaluations on a small set of public and private benchmarks. U.S. closed-weight models were evaluated with system-level safeguards disabled to reduce refusals and enable measurement of maximal capabilities. Publicly available versions of these models have these safeguards enabled. Due to the specifics of Kimi K3’s hosting setup, UK AISI / CAISI ran a selective set of cyber evaluations. Detailed methodologies are provided in individual sections.Cyber Capability TrendsThe cyber capability of models is aggregated across multiple tasks from multiple benchmarks using an approach inspired by Item Response Theory (IRT). For details of the methodology please see prior published reports.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Hacker News: Front Page.
Discussion
0 commentsMore from Hacker News: Front Page
