Center for Social Information Sciences (CSIS) Seminar

Abstract: Robust reinforcement learning (RL) focuses on designing optimal policies from data for MDPs with model uncertainties. Existing convergence guarantees for robust RL are either limited to tabular settings or use restrictive assumptions in the function approximation setting. We will present an RL algorithm for learning the optimal policy from data in the function approximation setting and provide finite time sample complexity bounds without requiring generative access to the underlying MDP model. Our algorithm uses a combination of ideas from distributionally robust optimization (DPO), two time-scale stochastic approximation, and traditional (non-robust) fitted value iteration and Q-learning.