Factorial Agent Markov Model: Modeling Other Agents' Behavior in presence of Dynamic Latent Decision Factors

Autonomous agents operating in the real world often need to interact with other agents to accomplish their tasks. For such agents, the ability to model behavior of other agents-both human and artificial-without complete knowledge of their decision factors is essential. Towards realizing this ability, we present Factorial Agent Markov Model (FAMM), a model to represent behavior of other agents performing sequential tasks. In contrast with most existing models, FAMM allows for behavior of other agents to depend on multiple, time-varying latent decision factors and does not assume rationality. To enable learning of FAMM parameters by observing behavior of other agents, we provide a set of variational inference algorithms for the unsupervised, semi-supervised, and supervised settings. These Bayesian learning algorithms for the FAMM enable agents to model other agents using execution traces and domain-specific priors. We demonstrate the utility of FAMM and corresponding learning algorithms using three synthetic domains and benchmark them against existing algorithms for modeling agent behavior. Our numerical experiments demonstrate that, despite the presence of multiple and time-varying latent states, our approach is capable of learning predictive models of other agents with semi-supervision.