Modeling Random Guessing and Task Difficulty for Truth Inference in Crowdsourcing

This paper addresses the challenge of truth inference in crowdsourcing applications. We propose a generative method that jointly models tasks' difficulties, workers' abilities and guessing behavior to estimate the truths of crowdsourced tasks, which leads to a more accurate estimation on the workers' abilities and tasks' truths. Experiments demonstrate that the proposed method is more effective for estimating truths of crowdsourced tasks compared with the state-of-art methods.