Sign-Separated Finite-Time Error Analysis of Q-Learning
The paper presents a sign-separated finite-time error analysis for constant step-size Q-learning. It decomposes the error into negative and positive components, revealing an asymmetry in Q-learning error dynamics. The findings indicate that negative errors are dominated by a linear time-invariant system, while positive errors are influenced by a linear switching system.
- ▪The analysis identifies a max-induced asymmetry in Q-learning error dynamics.
- ▪Negative errors are controlled by a lower comparison linear time-invariant system associated with a fixed optimal policy.
- ▪Positive action-wise errors can be selected and propagated by the Bellman maximum.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Artificial Intelligence arXiv:2605.16103 (cs) [Submitted on 15 May 2026] Title:Sign-Separated Finite-Time Error Analysis of Q-Learning Authors:Donghwan Lee View a PDF of the paper titled Sign-Separated Finite-Time Error Analysis of Q-Learning, by Donghwan Lee View PDF HTML (experimental) Abstract:This paper develops a sign-separated finite-time error analysis for constant step-size Q-learning. Starting from the switching-system representation, the error is decomposed into its componentwise negative and positive parts. The negative part is dominated by a lower comparison linear time-invariant (LTI) system associated with a fixed optimal policy, whereas the positive part is controlled by a linear switching system.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
