Attribute Based Object Recognition by Human Language

Over the last years, the robotics community has made substantial progress in detection and 3D pose estimation of known and unknown objects. However, the question of how to identify objects based on language descriptions has not been investigated in detail. While the computer vision community recently started to investigate the use of attributes for object recognition, these approaches do not consider the task setting typically observed in robotics, where a combination of colors, shapes, materials might be used in referral language to identify specific objects in a scene. In this paper, we introduce an approach for identifying objects based on natural language containing the attributes of the object. Our experiments show that by using the attributes mentioned in the referral language it is indeed possible to build a learning object detection system that does not require any training images of the target classes.