Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing
Unlike classical contextual bandits that rely solely on bandit feedback and assume conditional independence across arms, our setting allows context-dependent inter-arm correlations and auxiliary reward information that may be noisy or misspecified. We propose algorithms that leverage such surrogate rewards through two complementary designs. A coupled reward-mixing approach pools true and surrogate rewards to accelerate learning when surrogate signals are reliable, while a decoupled prediction-mixing approach maintains separate estimators for bandit feedback and surrogate rewards and adaptively combines their predictions.
- ▪Unlike classical contextual bandits that rely solely on bandit feedback and assume conditional independence across arms, our setting allows context-dependent inter-arm correlations and auxiliary reward information that may be noisy or missp
- ▪We propose algorithms that leverage such surrogate rewards through two complementary designs.
- ▪A coupled reward-mixing approach pools true and surrogate rewards to accelerate learning when surrogate signals are reliable, while a decoupled prediction-mixing approach maintains separate estimators for bandit feedback and surrogate rewar
Opening excerpt (first ~120 words) tap to expand
Computer Science > Machine Learning arXiv:2607.09015 (cs) [Submitted on 10 Jul 2026] Title:Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing Authors:Ajay Narayanan Sridhar, Ronak Singh, Mehrdad Mahdavi, Vijaykrishnan Narayanan View a PDF of the paper titled Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing, by Ajay Narayanan Sridhar and 3 other authors View PDF HTML (experimental) Abstract:We study contextual bandit problems with correlated arms and access to surrogate reward signals produced by a machine learning model, motivated by applications such as large language model (LLM) routing.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
