New enhancements to the DeepGaze models for a better understanding of human scanpaths

Abstract

The family of DeepGaze models comprises deep learning based computational models of freeviewing overt attention. DeepGaze II predicts freeviewing fixation locations (Kümmerer et al, ICCV 2017) and DeepGaze III (Kümmerer at al, CCN 2019) predicts freeviewing sequences of fixations. The models encode image information using deep features from pretrained deep neural networks to compute a spatial saliency map, which, in case of DeepGaze III, is then combined with information about the scanpath history to predict the next fixation. Both models have set the state of the art in their respective tasks in the last years. Here, we improve the performance of both models substantially. We replace the backbone deep neural network VGG-19 with better performing networks such as DenseNet. We also improve the architecture of the model and the training procedure. This results in a substantial performance …