Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs

Computer Science > Artificial Intelligence arXiv:2604.24395 (cs) [Submitted on 27 Apr 2026] Title:Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs Authors:Byeonggeuk Lim, JungMin Yun, Junehyoung Kwon, Kyeonghyun Kim, YoungBin Kim View a PDF of the paper titled Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs, by Byeonggeuk Lim and 4 other authors View PDF HTML (experimental) Abstract:Large Vision-Language Models (LVLMs) frequently suffer from hallucinations. Existing preference learning-based approaches largely rely on proprietary models to construct preference datasets. We identify that this reliance introduces a distributional mismatch between the proprietary and target models that hinders efficient alignment. To address this, we propose Alignment via VErified Self-correction DPO (AVES-DPO), a framework that aligns LVLMs using in-distribution data derived from the model's intrinsic knowledge. Our approach employs a consensus-based verification mechanism to diagnose diverse hallucinations and guides the model to self-correct, thereby generating preference pairs strictly compatible with its internal distribution. Extensive experiments demonstrate that AVES-DPO surpasses existing baselines in hallucination mitigation while requiring only 5.2k samples. Comments: Accepted to ACL 2026 Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2604.24395 [cs.AI] (or arXiv:2604.24395v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2604.24395 Focus to learn more arXiv-issued DOI via DataCite Submission history From: Byeonggeuk Lim [view email] [v1] Mon, 27 Apr 2026 12:22:35 UTC (2,016 KB) Full-text links: Access Paper: View a PDF of the paper titled Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs, by Byeonggeuk Lim and 4 other authorsView PDFHTML (experimental)TeX Source view license Current browse context: cs.AI < prev | next > new | recent | 2026-04 Change to browse by: cs References & Citations NASA ADSGoogle Scholar Semantic Scholar export BibTeX citation Loading... BibTeX formatted citation × loading... Data provided by: Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv (What is alphaXiv?) Links to Code Toggle CatalyzeX Code Finder for Papers (What is CatalyzeX?) DagsHub Toggle DagsHub (What is DagsHub?) GotitPub Toggle Gotit.pub (What is GotitPub?) Huggingface Toggle Hugging Face (What is Huggingface?) ScienceCast Toggle ScienceCast (What is ScienceCast?) Demos Demos Replicate Toggle Replicate (What is Replicate?) Spaces Toggle Hugging Face Spaces (What is Spaces?) Spaces Toggle TXYZ.AI (What is TXYZ.AI?) Related Papers Recommenders and Search Tools Link to Influence Flower Influence Flower (What are Influence Flowers?) Core recommender toggle CORE Recommender (What is CORE?) Author Venue Institution Topic About arXivLabs arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals…

This excerpt is published under fair use for community discussion. Read the full article at arXiv.org.