Using the DeepGaze III model to decompose spatial and dynamic contributions to fixation placement over time

Abstract

Humans view images in scanpaths of fixations, where they move their gaze over the image to explore interesting parts of the image. Which factors govern the principles of such scanpaths and how they change over time has been the subject of substantial research. The deep learning based DeepGaze III model currently sets the state-of-the-art in free-viewing scanpath prediction on natural images. It combines a spatial prediction module, which captures the influence of scene content on fixation placement, with a scanpath history module that captures the influence of earlier fixations and therefore the dynamics of the scanpath. Here, we conduct a series of ablation studies to train variants of DeepGaze III with no access to scene content, scanpath history or both and analyse how well fixations are predicted over the course of free-viewing scanpaths. We find that the overall predictability of fixations decays substantially …

Matthias Kümmerer
Matthias Kümmerer
Postdoc

My research interests include eye movements, saliency prediction, benchmarking, representation learning and statistics.

Matthias Bethge
Matthias Bethge
Professor for Computational Neuroscience and Machine Learning & Director of the Tübingen AI Center

Matthias Bethge is Professor for Computational Neuroscience and Machine Learning at the University of Tübingen and director of the Tübingen AI Center, a joint center between Tübingen University and MPI for Intelligent Systems that is part of the German AI strategy.