Comparing Search Strategies of Humans and Machines in Clutter

Abstract

While many perceptual tasks become more difficult in the presence of clutter, in general the human visual system has evolved tolerance to cluttered environments. In contrast, current machine learning approaches struggle in the presence of clutter. We compare human observers and CNNs on two target localization tasks with cluttered images created from characters or rendered objects. Each task sample consists of such a cluttered image as well as a separate image of one object which has to be localized. Human observers are asked to identify wether the object lies in the left or right half of the image and accuracy, reaction time and eye movements are recorded. CNNs are trained to segment the object and the position of the center of mass of the segmentation mask is then used to predict the position. Clutter levels are defined by the set-size ranging from 2 to 256 objects per image. We find that for humans …