Consistent Saliency Benchmarking: How One Model Can Win on All Metrics

Abstract

Understanding how humans place their gaze is important for understanding how humans exlore their environment and for computer vision applications and has attracted research for many decades. So called “saliency models” compute a “saliency map” to predict fixations for an image. Many different saliency models have been proposed, from low-level feature integration to complex deep-learning based models, and more are added every year.However the field is facing a fundamental problem: there is no agreed-upon metric for assessing the quality of a saliency map. Instead, eg the most commonly used MIT saliency benchmark evaluates a total of eight metrics which yield highly inconsistent model rankings. This has led to contradicting conclusions about which algorithms are most predictive. We have previously shown that treating models probabilistically and evaluating log-density saliency maps removes most …