Why AI Agents Cannot Change Software Systems
The article discusses the limitations of current LLMs in modifying real software systems. While they can assist with code generation, they lack the ability to understand complex dependencies and causal relationships necessary for safe modifications. This gap prevents LLMs from autonomously delivering software changes in production environments.
- ▪Current LLMs can assist with software delivery but cannot autonomously modify real software systems.
- ▪The distinction between additive and transformative tasks is crucial; LLMs can handle self-contained tasks but struggle with system-dependent changes.
- ▪Producing a safe pull request requires understanding the entire system, which LLMs are currently unable to do.
Opening excerpt (first ~120 words) tap to expand
This article explains why current LLMs cannot safely modify real software systems, despite impressive code‑generation demos. Table of contents The Promise of Automated Software Delivery In 2026, the automated software delivery dream is for an agent to: read a repository understand project structure plan a multi‑step change write code, tests, and docs run the code and fix its own mistakes produce a PR‑ready diff The first three tasks are additive; the last three are transformative. The first three add information without changing the behaviour of the system: they require reading, mapping, and planning, but not altering any existing causal structure in the codebase.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Phroneses.com.