Efficient Offline Reinforcement Learning Through Dataset Characterization and Reduction

Offline Reinforcement Learning (RL) relies heavily on the quality of datasets to derive effective policies. The dataset characteristics (such as Trajectory Quality or State-Action Coverage) impact the performance of the learned policy. Typically, these problems are solved by enhancing the algorithms used in the process (such as regularization methods and others). Despite recognizing these characteristics as crucial for determining the quality of a learned policy, what constitutes a "good" dataset remains ambiguous. This thesis explores methodologies for predicting the learning performance of offline RL datasets and optimizing their composition to improve policy outcomes. By representing datasets as images and using Convolutional Neural Networks (CNNs), we predict policy performance and enable efficient dataset reduction using genetic algorithms. Preliminary experiments demonstrate the potential for dataset size reduction while maintaining or enhancing policy quality. CCS CONCEPTS • Computing methodologies → Machine learning; Markov decision processes; Neural networks; Genetic algorithms.