Amdahl's Law for LLM generated code
Amdahl's Law highlights the limitations of using large language models (LLMs) for generating code. While LLMs can produce vast amounts of code, ensuring its correctness requires thorough human auditing, which is often more complex than writing the code itself. This suggests that the potential efficiency gains from LLMs are fundamentally constrained when it comes to critical coding tasks.
- ▪LLMs can generate millions of lines of code, but correctness is not guaranteed.
- ▪Hand-auditing every line of code is necessary for important projects.
- ▪The speedup from LLMs is limited when writing code that requires careful consideration.
Opening excerpt (first ~120 words) tap to expand
LLMs may theoretically be able to generate millions of correct lines of code.But for any important code the only way to know that it's correct is to hand-audit every line, which is harder than writing every line because this allows you to build a conceptual hierarchy and model within which you can think about the code.So there's a fundamental limit to the speedup from LLMs when writing anything you care about.(The main objection will be what about when you have a human engineer working on your behalf. But this is a distraction: sometimes you can fully trust an engineer, which is not at all true for LLMs.)
Excerpt limited to ~120 words for fair-use compliance. The full article is at Ycombinator.