Approximating Difference Evaluations with Local Knowledge

Difference evaluation functions have resulted in excellent multiagent behavior in many domains, including air traffic control and distributed sensor network control. In addition to empirical evidence, there is theoretical evidence that suggests difference evaluation functions help shape private agent utilities/objectives in order to promote coordination on a system-wide level. However, calculating difference evaluation functions requires global knowledge about the system state and joint action as well as the mathematical form of the system objective function, which are often unavailable. In this work, we demonstrate that a local estimate of the system evaluation function may be used to locally compute difference evaluations, allowing for difference evaluations to be computed in multiagent systems where only local state and action information as well as a broadcast value of the system evaluation function are available. We demonstrate that approximating difference evaluation functions results in better performance and faster learning than when using global evaluation functions, and performs only slightly worse than when directly computing difference evaluations.