Claude Code Degraded Before Opus 4.8 Release
Claude Code experienced a significant drop in its pass rate prior to the release of Opus 4.8. This decline lasted for five consecutive days before recovering immediately after the new version was launched. The performance tracking highlighted the importance of monitoring day-to-day changes in model performance.
- ▪Claude Code's pass rate dropped below its established baseline before the Opus 4.8 release.
- ▪The degradation lasted for five days and was statistically significant.
- ▪The pass rate rebounded sharply once Opus 4.8 was released on May 28.
Opening excerpt (first ~120 words) tap to expand
Back to blog claude-code degradation swe-bench-pro Claude Code degraded for the week before Opus 4.8's release Our SWE-Bench-Pro tracker caught a statistically significant, weeklong drop in Claude Code's pass rate just before Opus 4.8 shipped, and the recovery that followed. JB · May 28, 2026 We run Claude Code against a curated subset of SWE-Bench-Pro every day, in the plain CLI with the current SOTA model and no custom harness. The point is to catch the thing that benchmarks published at launch can’t: silent, day-to-day changes in how a model actually performs in production. In the week leading up to the Opus 4.8 release, the tracker caught exactly that.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Marginlab.
Discussion
0 commentsMore from Marginlab
