Learning to Explore when Mistakes are Not Allowed

Goal-Conditioned Reinforcement Learning (GCRL) enables learning unified controllers but its trial-and-error process poses risks in real-world applications. We propose a method that allows agents to explore while avoiding harmful mistakes. Since environment dynamics are often uniform in space, a policy trained for safety without exploration purposes can still be exploited globally. Our approach has two phases: first, pretraining a safety policy using safe reinforcement learning and distributional techniques; second, ensuring safe exploration by selecting the action to perform on the environment either from the safety policy or from the learning goalconditioned (GC) policy, depending on current state. In simulated environments, we show that it covers most of the goal-space while minimizing mistakes during exploration, unlike traditional GCRL. We also perform an ablation study and failure analysis, providing insights for future research.