LLMs require curated context for reliable political fact-checking
A recent study highlights the limitations of large language models (LLMs) in political fact-checking. While reasoning capabilities and web search tools have been integrated into mainstream chatbots, their effectiveness remains questionable without curated context. The research indicates that providing LLMs with high-quality, curated information significantly enhances their fact-checking performance.
- ▪Standard models of LLMs perform poorly in political fact-checking tasks.
- ▪Reasoning capabilities offer minimal benefits, while web search provides moderate improvements.
- ▪A curated retrieval-augmented generation (RAG) system using PolitiFact summaries improved performance by 233% on average across model variants.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Computation and Language arXiv:2511.18749 (cs) [Submitted on 24 Nov 2025] Title:Large Language Models Require Curated Context for Reliable Political Fact-Checking -- Even with Reasoning and Web Search Authors:Matthew R. DeVerna, Kai-Cheng Yang, Harry Yaojun Yan, Filippo Menczer View a PDF of the paper titled Large Language Models Require Curated Context for Reliable Political Fact-Checking -- Even with Reasoning and Web Search, by Matthew R. DeVerna and 3 other authors View PDF Abstract:Large language models (LLMs) have raised hopes for automated end-to-end fact-checking, but prior studies report mixed results.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv.org.
Discussion
0 commentsMore from arXiv.org
