Characterizing and Aggregating Agent Estimates

In many applications, agents (whether human or computational) provide estimates that must be combined at a higher level. Recent research distinguishes two kinds of such estimates: interpreted and generated data. These two kinds of data require different kinds of aggregation processes, which behave differently from an information geometric perspective: interpreted estimates require methods such as voting that can leave the convex hull of the individual estimates, while the optimal aggregation for generated estimates lies within the convex hull and thus is accessible by methods such as weighted averages. We motivate our analysis in the context of a crowdsourced forecasting application, demonstrate the central insights theoretically, and show how these insights manifest themselves in actual data.