Making and Improving Predictions of Interest Using an MDP Model

In many cases, building a generative model is difficult and unnecessary since we may be only interested in making predictions of some certain situations. In this paper, we model the dynamics of predictions of interest, called prediction profile, through a Markov decision process (MDP) and accordingly make the predictions using the learned model. We further adapt the entropy concept to measure prediction accuracy of the learned MDP model and provide important guidelines for strategically expanding events of interest with the purpose of improving the predictions. We conduct experiments to demonstrate the performance of our techniques.