Valuing Knowledge, Information and Agency in Multi-agent Reinforcement Learning: A Case Study in Smart Buildings

Increasing energy efficiency in buildings can reduce costs and emissions substantially. Historically, this has been treated as a singleagent optimization problem. However, many buildings utilize the same types of thermal equipment e.g. electric heaters and hot water vessels. During operation, occupants in these buildings interact with the equipment differently thereby accelerating state-space exploration. Reinforcement learning agents learn from these interactions, recorded as sensor data, to optimize the overall energy efficiency. However, if these agents operate at a household level, they can not exploit the replicated structure in the problem. In this paper, we demonstrate that multi-agent collaboration can improve control by exploring the state-space better. We also investigate trade-offs between integrating human knowledge and additional sensors. Results show that savings of over 40% are possible with collaborative multi-agent systems making use of either expert knowledge or additional sensors with no loss of occupant comfort. We find that such multi-agent systems outperform comparable single agent systems.