From Stochastic Bandits to Kernel Function Approximation in Offline Reinforcement Learning
吳沛遠副教授臺灣大學電機工程學系 15:30 – 17:00 Gedung Hongjing M107Abstract:
In this tutorial, we begin with stochastic bandits, where a learner chooses an action each round, observes only that action’s noisy reward, and aims to minimize cumulative regret relative to always choosing the action with the highest expected reward. We use this setting to introduce optimism in the face of uncertainty and review prior work on Linear UCB and its regret guarantees, Kernel UCB for nonlinear rewards, and LSVI and LSVI-UCB for value estimation and exploration in reinforcement learning. We conclude by connecting these ideas to our lab’s research on kernel Function Approximation in Offline Reinforcement Learning. Standard offline RL learns from a fixed dataset of transitions with recorded rewards. Our work studies how kernel methods can use both labeled and unlabeled transitions to learn a policy when reward labels are available for only part of the dataset.