NL-PAC: Specification Ambiguity and Certified Minimax Risk Floors in LLM-Mediated Supervision
When a specification admits multiple readings but the supervision channel does not reveal which is operative, additional labels reduce sampling error without resolving the resulting identification problem. We introduce Natural Language PAC (NL-PAC), a framework that uses a fixed model's thresholded decoding law to define admissible labels and candidate targets. Finite-sample confidence bounds make these quantities certifiable from held-out unlabeled inputs.
- ▪When a specification admits multiple readings but the supervision channel does not reveal which is operative, additional labels reduce sampling error without resolving the resulting identification problem.
- ▪We introduce Natural Language PAC (NL-PAC), a framework that uses a fixed model's thresholded decoding law to define admissible labels and candidate targets.
- ▪Finite-sample confidence bounds make these quantities certifiable from held-out unlabeled inputs.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Machine Learning arXiv:2607.08961 (cs) [Submitted on 9 Jul 2026] Title:NL-PAC: Specification Ambiguity and Certified Minimax Risk Floors in LLM-Mediated Supervision Authors:Berkay Anahtarci View a PDF of the paper titled NL-PAC: Specification Ambiguity and Certified Minimax Risk Floors in LLM-Mediated Supervision, by Berkay Anahtarci View PDF HTML (experimental) Abstract:Large language models increasingly provide labels, evaluations, and feedback for tasks specified in natural language. When a specification admits multiple readings but the supervision channel does not reveal which is operative, additional labels reduce sampling error without resolving the resulting identification problem.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
