Improving Sample-based Reinforcement Learning through Complex Non-parametric Distributions

Sampling-based approaches in Reinforcement Learning (RL) typically involve learning or maintaining distributions. While many elegant algorithms were proposed in literature, most methods involve prior assumptions of the underlying distributions (eg. being Natural Exponential Family), or the number of modality for either simplicity or tractability reasons. A method to effectively apply complex or non-parametric distributions, for example, distributions approximated using neural network, is still lacking. One example is the limitation of using of reparameterized Gaussian policy, rather than any arbitrary non-parametric policy in Soft Actor-Critic (SAC) amenable to the necessary entropy estimation. The thesis would be focusing on proposing and evaluating methods to enable better approximation of complex distributions, and methods to estimate measurements of non-parametric distributions. The motivation is to allow better connections and applications of many deep learning and information theory techniques to the samplingbased approaches in RL, by alleviating limitations and difficulties of complex non-parametric distributions.