Optimal Sequential Planning for Communicative Actions: A Bayesian Approach

We build on the Interactive POMDP (IPOMDP) framework, which extends POMDPs to multi-agent settings, and include communication which can take place between the agents. While IPOMDPs endow the agents with models of their environments and models of other agents, we supplement IPOMDPs with communicative acts available to the agents to formulate Communicative IPOMDPs (CIPOMDPs). We treat communication as a type of action; hence decisions regarding communicative acts should be based on decisiontheoretic planning using Bellman optimality principle, just as they are for all other actions. As in any form of planning, the results of actions need to be precisely specified. We use Bayes update to derive how agents update their beliefs in CIPOMDPs; updates are due to their actions, observations, messages they send to other agents, and messages they receive from others. Without communication CIPOMDPs reduce to IPOMDPs. Without other agents they all become classical POMDPs.